DEGREE PROGRAMME IN WIRELESS COMMUNICATIONS ENGINEERING

MASTER’S THESIS USABILITY OF MULTI-AGENT SIMULATORS IN SIMULATION OF WIRELESS NETWORKS Author Kumud Bandav Wasti Supervisor Docent Harri Saarnisaari Second Supervisor Doctor Timo Bräysy

October, 2014

Wasti B.K. (2014) Usability of Multi-Agent Simulators in Simulation of Wireless Networks. University of Oulu, Department of Communications Engineering, Master’s Degree Program in Wireless Communications Engineering. Master’s thesis, 82 p.

ABSTRACT Multi agent system (MAS) is considered an effective way to design, analyze and implement complex systems in various disciplines nowadays. However, MAS has not been sufficiently applied in mainstream modelling of wireless communication networks yet. The exponential growth of network users has led today’s communication networks even more significantly complex. As such, modelling and simulation tends to grow in proportion but not markedly to meet the preciseness and complexity of the system under consideration. Furthermore, the future mobile communication seems to incline over self-adaptive resource allocation and congestion control mechanisms. This aims to relieve the workload of the network operator while maintaining the QoS level of the communication intact. The advancements also envision the autonomous nature of future wireless communication technologies where dynamic resource allocation will play a vital role in efficient bandwidth utilization, throughput maximization and effective power control mechanisms. In this context, MAS brings about various possibilities to model and simulate environments which are typically stochastic in nature and tends to grow in a decentralized manner.

MAS based simulation is such a simulation method which has gained recognition in modeling telecommunication systems where issues like network complexity, traffic and network management are prevalent. MAS has its proven standing to represent a real interactive environment of a system where learning, negotiation and cooperation are important phenomena. On that account, MAS has already started to establish its turf in the wireless communication systems as well. As a matter of fact, the existing network simulators are not designed to incorporate the features that MAS has to offer. So, the agent based simulation toolkits are used alongside, on top of which most of the physical environment aspects can be modeled that are usually neglected by the existing network simulators. This thesis investigates the areas of wireless communications where multi agent systems and simulators are applicable by the means of literature study and presents simple simulation models constructed in Netlogo, a multi agent based simulation toolkit.

As a result, the applicability of MAS simulators in areas like radio channel environment, resource allocation, routing, medium access control and cognitive radio is presented. At the end, conclusions are drawn about feasibility of using MAS simulators to simulate wireless communication networks.

Keywords: simulations, multi agents, wireless communication, power allocation, cognitive radio, Netlogo, Repast.

TABLE OF CONTENTS ABSTRACT TABLE OF CONTENTS PREFACE LIST OF ABBREVIATIONS 1. INTRODUCTION ........................................................................................... 08

1.1. Motivation ............................................................................................... 08 1.2. Objectives ................................................................................................ 09 1.3. Report outline .......................................................................................... 09

2. BASIC CONCEPTS OF MAS ....................................................................... 10

2.1. Introduction to MAS ................................................................................ 10 2.2. Key concepts and terms ........................................................................... 11

2.2.1. Agents and their taxonomy .......................................................... 11 2.2.2. Environment ................................................................................ 12 2.2.3. Communication ............................................................................ 13 2.2.4. Roles ............................................................................................ 14 2.2.5. Resources ..................................................................................... 14

2.3. Application areas of multi agent systems ................................................ 14 2.4. Strengths and limitations of MAS ........................................................... 16

3. SIMULATIONS WITH MAS ........................................................................ 18

3.1. Multi agents simulators and modeling practices ..................................... 18 3.2. MAS system model, design and implementation .................................... 19

3.2.1. Agent architecture and behavior design ...................................... 20 3.2.2. Environment design ..................................................................... 21 3.2.3. Running the model ....................................................................... 23 3.2.4. Testing and verification ............................................................... 23

3.3. Some existing multi agent based simulators ........................................... 24 4. COMMUNICATION STANDARDS IN MAS BASED DISTRIBUTED

SIMULATIONS..............................................................................................277 4.1. Communications based on FIPA ............................................................. 27

4.1.1. FIPA-ACL ................................................................................... 28 4.1.2. Interaction protocols .................................................................... 30

4.1.3. Content language ......................................................................... 30 4.2. Knowledge query and manipulation language (KQML) ......................... 30

5. USABILITY OF MAS IN COMMUNICATION NETWORKS AND

COGNITIVE RADIO………………………………………………………...33 5.1 Simulation model design for communication networks………………...33 5.2. Radio Access Technology (RAT) ........................................................... 34

5.2.1. Simulation model design for radio environment………………..35 5.2.1.1 Functionality of the model……………………………………...37

5.2.1.2 Mobility in the model…………………………………………...40 5.3. Resource management and load balancing in wireless networks ............ 41 5.4. Applicability of routing algorithms with MAS ....................................... 45 5.5. Modeling Medium Access protocols ....................................................... 50

5.5.1. Simulation model for medium access .......................................... 51 5.6. Modeling Cognitive features ................................................................... 55 5.6.1. Simulation model for slot assignment based on priority………..57

6. SUMMARY ..................................................................................................... 60 7. REFERENCES .................................................................................... ………63 APPENDIX A ……………………………………………………………………...72 APPENDIX B ……………………………………………………………………....79 APPENDIX C ………………………………………………………………………80

FOREWORD This Master’s thesis has been carried out at the Department of Communication Engineering, University of Oulu, Finland. The aim of this thesis is to explore and present the applicability of multi agent simulators for the study of wireless communication networks. I would like to express my sincere gratitude to my supervisors Docent Harri Saarnisaari and Doctor Timo Bräysy for their continuous support and guidance throughout my work. I would also like to thank my supervisors for encouraging me in every step of my difficulties during this thesis. My deepest appreciation goes to my family and friends for their unconditional support and motivation throughout my studies. Last but not the least; I would like to present my deepest thanks to the University of Oulu and all the members for their contributions towards creating a very comfortable environment for learning and thriving.

Oulu, October 21, 2014 Kumud Bandav Wasti

LIST OF ABBREVIATIONS ABMS Agent Based Modeling and Simulation ACC Agent communication channel ACL Agent Communication Language ACM Agent Communications Medium AI Artificial Intelligence API Application Programming Interface ASCII American Standard Code For Information Interchange BDI Belief Desire Intention BER Bit Error Rate BS Base Station BTS Base Transceiver Station (GSM - UMTS) C/N Carrier To Noise Ratio CCC Common Control Channel CNIR Carrier to Noise Plus Interference Ratio CR Cognitive Radio CRAHNs Cognitive Radio Ad Hoc Networks CRN Cognitive Radio Network CRNs Cognitive Radio Networks CSMA Carrier Sense Multiple Access CSMA/CA Carrier Sense Multiple Access With Collision Avoidance CSMA/CD Carrier Sense Multiple access With Collision Detection CTS Clear To Send DAI Distributed Artificial Intelligence DBSCAN Density-Based Spatial Clustering of Applications with Noise DL Downlink DMP Dynamic Mobile Probability Eb/No Energy per Bit to Noise Spectral Density Ratio EIRP Effective Isotropic Radiated Power (Radio) FIPA Foundation For Intelligent Physical Agents FSPL Free Space Path Loss GIS Geographic Information System GPL General Public License GSM Global Standard/System For Mobile Telecommunications HPC High Performance Computing IEEE Institute Of Electrical And Electronics Engineers IIOP Internet Inter-ORB Protocol IMTP Internal Message Transport Protocols JADE JAVA Agent Development Framework JVM Java Virtual Machine KIF Knowledge Interchange Format KQML Knowledge Query and Manipulation Language LANs Local Area Networks

LOS Line Of Sight MABS Multi Agent Based Simulation MAC Media Access Control (IEEE 802) MANET Mobile AdHoc Network MAS Multi Agent System NLOS Non Line Of Sight NS-2 Network Simulator-2 PDF Probability Distribution Function PRS Procedural Reasoning System QoS Quality Of Service RAN Radio Access Network RAT Radio Access Technology (GSM - UMTS) RF Radio Frequency RMI Remote Method Invocation ROM Read Only Memory RRM Radio Resource Management RSS Received Signal Strength RSSI Received Signal Strength Indicator RTS Request To Send (EIA-232-E) RTS/CTS Request To Send/Clear To Send SDR Software Defined Radio SINR Signal to Interference Noise Ratio SMTP Simple Mail Transfer Protocol (IETF) SNR Signal-To-Noise Ratio SS7 Signaling System 7 (ANSI) STDMA Spatial Time Division Multiple Access SUs Secondary Users SYNC Synchronize TCP/IP Transmission Control Protocol /Internet Protocol (IETF) TDMA Time Division Multiple Access UE User Equipment (GSM - UMTS) UWB Ultra Wide Band WiMax Worldwide Interoperability for Microwave Access WLAN Wireless Local Area Network WSNs Wireless Sensor Networks XML Extensible Markup Language

8

1. INTRODUCTION The design and implementation of communication network involves a rigorous study and procedures to be followed. But without an estimation of how the network would perform in the real world and what behavior it would exhibit in the physical scenario, the implementation of a communication network would lack confidence. This is even true if a system is based on a complex mathematical model and the network that evolves through it, bear dynamic attributes. Furthermore, a communication system may be a realizable one or may it be just an abstract concept which is yet to be developed, both of these contexts need a platform to flourish in which a designer can test, modify and evaluate the protocols that govern the communication networks.

1.1. Motivation

Wireless communication network is a realm of communication which is dynamic in nature because of the environment in which it operates. Mobility, limited bandwidth, latency, interference from neighboring networks and topology changes are the major challenges that make wireless networks more unpredictable and complex. The design of such a dynamic system requires a greater scrutiny than the wire-line systems. Thus, simulations comes forward as such a platform in which a wireless network engineer can do the preliminary study of the behavioral aspects of the network under various conditions by modifying the attributes of the wireless environment, just by sitting on a lab. Each discipline of technology has its own set of standards and hence the simulation platforms also vary from discipline to discipline. As such, there exists numerous simulators used to model wireless networks such as OPNET [1], NS-2 [2], OMNeT++ [3] to name a few. Nevertheless, it is important to note that, the wireless communication paradigm has shown an evolving change with respect to system architecture, communication methodologies, bandwidth needs, data rates, and topology in past few years. Concepts of decentralization, cognitive radio and even software radio based technologies like GNU radio [4], OpenBTS [5] etc have sprung up with promising features to avail today’s communication needs. As the technology advances, so do the network complexity and management issues. But the simulators have not evolved to keep pace with these changing technologies.

It is worth to mention that the application of multi agent systems (MAS) in various layers of communications has a successful outcome in the overall system’s performance. MAS can be seen as the collection of autonomous entities that bear attributes to perceive the environment, decision distribution and self organization. These attributes are mandatory in the cognitive radio (CR) terminals and thus the MAS finds its place in modeling CR networks. However, cognitive radio is not the only area MAS has gained a reputation. MAS is also exploited every now and then in distributed networks like wireless ad-hoc and sensor networks for routing, medium access control (MAC) functionalities and resource allocation problems in cellular networks.

This research is mainly motivated by the study of MAS in details and explore the agent based modeling and simulation (ABMS) tools to model the areas of wireless communication networks.

9

1.2. Objectives

The goal of this thesis is to research the multi agent systems in details, usability of MAS in the sectors of wireless communication and applicability of ABMS tools to simulate wireless communication networks. This will be accomplished with the help of literature studies and building simple simulation models in a selected multi agent based simulator called Netlogo [6]. Furthermore, this research is also focused on the study of the potential benefits, the ABMS tools can offer and its limitations in modeling wireless networks.

1.3. Report outline

The report begins with basic concepts of multi agent systems, application areas and its strength and weakness in Chapter 2. Some existing multi agent simulators are presented and MAS simulation practices are discussed in Chapter 3. Chapter 4 introduces the communication standards and languages used in distributed MAS simulations. Chapter 5 is dedicated to provide the details of MAS based simulation in wireless communication networks with three simulation models and literature reviews while conclusion and some possible future works are discussed in Chapter 6.

10

2. BASIC CONCEPTS OF MAS This chapter provides the fundamentals of multi agent systems, the benefits and its drawbacks. Section 2.1 introduces multi agent systems with definitions. Some key concepts and elements of MAS are elaborated in section 2.2. Section 2.3 describes the application areas of MAS whereas in section 2.4, strength and weakness of multi agent based systems is discussed.

2.1. Introduction to MAS

The idea of agent is often found to be associated with the domain of artificial intelligence (AI) [7] since its conception. Agent was then used to study and develop the machine intelligence. The independent machines were autonomous in nature and the primary objective for such intelligent entities was the simulation of human reasoning, involving only one actor scenario.

With time, the demand of computation changed from centralized to distributed nature. The complexity of the system also increased accordingly. This gave rise to the idea of distributed artificial intelligence (DAI) [8] and hence the era of MAS was born [9], shifting from individual to collective models. Multi agent systems being one of the core research areas of DAI, it is used extensively as a simulation tool to model the distributed intelligence of a complex system, where each intelligent agent is capable of functioning autonomously, in parallel with other functional agents and with cooperation whenever needed. Definition of MAS: Multi-agent systems may be defined as an assembly of distributed autonomous entities that are intelligent and capable of accomplishing complex tasks by the virtue of interaction, coordination and emergent collective behavior [11,12]. These entities interact with each other according to the rules that govern their objectives and the goal of the overall system. They follow a particular task continuously and are of intelligent in nature. In the MAS paradigm these entities are called agents. These agents work collectively to carry out certain goal provided to them by the observer (user) in a complex system. Commonly driven by belief-desire-intention architecture (BDI) among cognitive agents, MAS uses this software model to program an agent in order to solve a particular problem. The architectural components of this model are: Belief: It is the repository of the agent where information or status of the environment, other agents and itself is stored. Desire: Desire is the eventual goal an agent is programmed for. This is the target an agent is supposed to achieve. Intentions: Intention is the series of actions an agent performs in an environment. It is the intended plan it has to perform to complete a goal set. A sample algorithm for the

11

main interpreter loop in a procedural reasoning system (PRS) is presented in Algorithm 1 [12]. The algorithm basically explains how an intention structure is built, executed and modified in the BDI based architecture. Algorithm 1 A sample BDI interpreter algorithm 1) initialize-state 2) repeat options: option-generator(event-queue) selected-options: deliberate(options) update-intentions(selected-options) execute() get-new-external-events() drop-unsuccessful-attitudes() drop-impossible-attitudes() 3) end repeat

A Multi agent system is developed for the simulation of large service systems which are focused on the execution in orders and the realization of services related to them. In simulations involving multi agent systems, a MAS is used as a model to represent the real world pattern such as cooperation in a communication network that may include nodes with various identities, all interacting with each other in a diverse and complex manners. These nodes are basically the agents that have a certain level of independency but they must make decisions jointly as a group in multi agent decision making systems. Next section describes the building blocks of multi agent systems.

2.2. KEY CONCEPTS AND TERMS

2.2.1. Agents and their taxonomy

Agents are the physical or virtual entities that are placed in a particular environment and are able to take independent actions according to the feedback received from its surroundings. This ability of agents makes them autonomous and they execute their individual goals at the same time while sharing the common resources in a system. An agent can be a software program running in parallel computers, or set of programs in a single local computer or even a hardware model in a testbed, for instance a robot sensing and making decisions based on the environment variables. Agents are characterized by their features such as being autonomous, exhibiting social behaviour, re-activeness and pro-activeness.

Although various classification of agents are found in literature [13,14,15,16], they often seem to be contentious to each other on the basis of definitions. However,

12

agents may be purely of one breed or a hybrid or even heterogeneous types, all existing in a common environment, they often represent a collaborative goal in that system. In this regard, the agents can be classified in general as in Figure 1.

Figure 1. General classification of agents.

i) Reactive agents [15]: They are such type of agents which possesses no goal at all but still plays an important role in the simulation. These types of agents wait until an instruction is given to accomplish a task and does not get involve in decision making unless it is told to. If we look at a communication simulation scenario, such agents could be obstacles in between the communication paths, physical objects etc. ii) Proactive agents [15]: They are the agents which have a simple goal in a model and involve basic calculation and programming. They are the decision making entities which are often interactive in nature so as to gather information from the network. A bird flying in random directions in bird flocking model could be an example for such agents. iii) Cognitive agents [17]: They are complex agents embedded with intentions that take actions by sensing the environment. They are the intelligent blocks of a system that responds to the changes in the environment through learning capabilities. Cognitive agents are autonomous in nature and usually possess adaptive reasoning skill sets. They are capable of decision making and require complex calculation and programming as dynamics is involved. A sensor node in wireless sensor network that features power management through cooperation could be one of the examples of cognitive agents.

2.2.2. Environment

Environment is the shared and structured platform used by the agents to perform its goal. The environment could be thought as a virtual world which may or may not have implicit or explicit effects on the agent. An environment represents the geographical space in which a model is formed. Such an environment is called spatially explicit environment. While developing models, coordinates are used to point a location of an

Agents

Reactive Proactive Cognitive

13

agent in this virtual world. Figure 2 represents the agent functionality in the environment. Agents perceive the environment through sensing units and act upon the environment with units called actuators.

Actuator

Sensing

Decision making

Solving

Communication

Objective/Task assignment

Environment

Environment

Figure 2. Agent functionalities in an environment.

2.2.3. Communication

In order to exchange information and perform coordination, agents must communicate with each other relying on some sets of pre-defined protocols such as foundation for intelligent physical agents (FIPA) [18] request protocol, query protocol and contract net protocol. The communication between agents follows three main methods: agent-to-agent, agent-to-agents and through a relay agent. These concepts of communication protocols and agent communication language are elaborated in the Chapter 4. Message passing and blackboard systems are two communication mechanisms found in MAS communication. The message passing mechanism refers to the communication method where an agent passes a message to another agent directly, using the environment solely for transportation purpose. This types of messages are constructed based on speech act theory [19] . This theory has been originated in the field of linguistics and philosophy of language that defines how a speaker uses the language for completion of his intended action and how the receiver interprets it. Another form of communication method is the blackboard system. The black board is a architectural model used in artificial intelligence. It is a common knowledge base used by the participating agents to frequently update it with information or data that can be used by other agents for various

14

purposes. This becomes an indirect form of communication between agents but a versatile method of sharing knowledge, problem solving and resolving conflicts between agents which arises due to agents performing the same task at the same moment of time.

2.2.4. Roles

A role is a set of attributes and behavior of an agent that makes it unique in a group of agents. These attributes are acquired by an agent so that it has a unique goal in an environment. The role provides an explicit character to the agent that binds the agent to its environment. This character not only provides an agent its identity but also makes an agent reusable. The roles can be dynamic that changes with time or the role can be of short life span that drops off after the completion of agent’s goal.

2.2.5. Resources

A resource in multi agent system is the means of achieving an intended outcome. Such resources are utilized by agents to survive in particular phenomena thereby fulfilling an objective that is expected by the agent. Resources are usually of limited availability and are the matter of contention when two or more agents need the common resource at the same instant and at the same place. But behaviors like cooperation and coordination between agents make up the resource utilization efficient. The characteristics of a resource are totally based on a system. From an observer’s point of view, the resource may be means to undertake an action that produces an output or it may be just a factor of performing analysis inside a multi agent system.

2.3. Application areas of multi agent systems

The conception of MAS dates back to 1940s when John Von Neumann proposed the cellular automata [20], which is a discrete model studied in the fields of physics, mathematics, biology, microstructure modeling etc. In those times, the intense computational requirements for modeling such complex systems were not available and hence the concepts of MAS remained only abstract. But as of 1990s the technology grew and the potential of computing machines provided suitable platforms to build agent based models. Modeling software and languages became wide spread and application of MAS gained popularity in social sciences such as trade networks and demographic studies, artificial intelligence, biological science, military, logistics, traffic management etc [21]. With a long history of evolution, MAS has become a popular mode of study in communication network for dynamic load balancing, scalability, cognitive radios and so on. In sociology multi agent based models are often found to be adopted in computational sociology to study the social life by modeling the interactions between the adaptive agents and the impact of those interactions on a collective organization [21,22]. Mathematical representation of real world situation, challenging the established

15

theory, study of mathematical models of conflict and cooperation such as game theory [24], graph theory, combination and permutation, probabilistic and statistical analysis and mathematical optimization are the major areas of MAS application in mathematics. An extensive use of MAS can be found in the field of biology including the analysis of epidemic spread, cells organization, disease control, animal behavior, bio-informatics biomedicine and ecosystem management. In economics, MAS is used to study the transaction behaviors between traders in e-commerce that involves reasoning and negotiation, distributed nature of trading in financial markets, stocks and other financial sources analysis. In internet and computers, MAS is used in online auctions, internet shopping, cloud computing, data mining, monitoring and surveillance and many others. Although the use of MAS is very diverse, Figure 3 represents the application domain of MAS that are more common.

Figure 3. Application areas where MAS is used along with the major fields of interest.

MAS

Economic Theory-Markets

Distributed systems-Communication andcooperation

Psychology-Character

Telecommunications-Wireless sensornetworks,Cognitiveradio.

Biology-Ecology andenvironment

Mathematics-Optimizationproblems

Sociology-Organizationalbehaviour

16

2.4. Strengths and limitations of MAS

The strengths and drawbacks of using multi agent systems are outlined below.

Strengths:

Parallel computation and robustness: A system’s operation could speed up with parallel computation as multiple agents are deployed, each with tasks that are different from each other. Another merit of using MAS in load balancing systems [25] is that, multiple instances of agents can remove single point of failures thus improving a system’s performance by adding robustness to the system. Flexibility: Independent tasks and time bounded requirements can be handled easily and efficiently. Redundancy: Fault tolerance can be boosted due to redundant implementation of agents and because the task is shared. Granularity: From a programmer’s point of view, the granularity of MAS simulators results in relatively simple programs. Scalability and reusability: Adding new agent is relatively easy to the systems whose capabilities and functionalities are expected to vary. Agents that are designed to be independent and have unique characteristics can be reused in other systems which is similar to using a particular library in different frameworks. Independency: Since the repetitive tasks could be automated, MAS simplifies the human-machine interaction level by cooperation between autonomous agents rather than a centralized control. Limitations:

Agent based simulations are relatively complex because every interaction between

the agents and environment has to be defined each time an agent and environment variables are designed.

A high computational requirement is one of the drawbacks when extensive computation is required with large number of agents involved and with a complex environment.

The development time in a multi agent system model could be high due to the fact

that the components of MAS require a clear demarcation between themselves while designing. This becomes monotonous to the developer of the model.

17

Insufficient mechanism for representation of organizational structure [26]. But this drawback is not convincing as many authors have proposed several mechanisms and organizational structures such as the Gaia methodology, AALAADIN, etc [26,27]

18

3. SIMULATIONS WITH MAS

This chapter provides the simulation methodologies in multi agent systems. First a brief background of MAS simulators is presented, followed by the explanation of underlying techniques used in the development of agent based simulation models. Some existing MAS simulators are also considered with a wireless communication point of view at the latter part of the chapter.

3.1. Multi agents simulators and modeling practices

Multi agent simulators are used to simulate the interactions of agents (regardless of whether it is intelligent or not) with an aim to assess their behavior impacting the overall system. It is especially beneficial to model systems that cover distributed functionalities, and emergence and evolution is the progressive pathway to depict the actual organizational behavior of the real world domain. MAS simulators integrates the components of complex systems, evolutionary programming, graph theory, game theory, statistics, stochasticity, emergence and computational economics. Iterative random sampling methods such as Monte Carlo simulations techniques are used to consolidate the results and MAS simulators have their own tools for data representation and graphics.

Multi agent systems can be simulated using the general, all purpose softwares or programming languages or it can be simulated with softwares and languages that are specifically designed with multi agent functionalities. The choice of the toolkit or programming language is often based on the user’s knowledge and expertise. Although, the characteristics of the system to be modeled and to what scale the model will grow, can also be the influential factors for determining the platform. Therefore, according to these factors, a general classification of these simulators can be deduced as in Figure 4. However, a comparative analysis of the most popular toolkits and platforms are discussed in the section 3.5.

Figure 4. Classification of MAS modeling platforms based on user preference.

Small scale agent oriented + general

computation softwares

Netlogo, Repast S,Jason,

Mason

Matlab, OMNet++,

OPNET, JSim

Large scale agent oriented

Repast HPC, Swarm

Anylogic, Jade, Pandora

All purpose programming language

Java, Objective C, C, C++

Scala, Python, Erlang, .NET

19

Multi agent oriented toolkits like Netlogo [6] and Recursive Porous Agent Simulation Toolkit (Repast) Simphony [29] are designed to get a quick hand for first time users. They have relatively simple architecture and visual interactive interface with point-and-click development environment. Netlogo uses a spirit of Logo programming language [30] whereas both Java [31] and Relogo [32] languages can be used in Repast Symphony for modeling. They have their own in-built results collection and graphical plotting tools with automated connection to other tools like R [33], Matlab [34] etc for generating richer graphic plots. But it should also be noted that these simulators do not include functions to work on polar and spherical coordinate system and hence polar plots are not provided along with the system and the extensions are not available till date. This would otherwise be very useful for simulating features like antenna directionality and wave propagation models. Another useful feature namely complex number calculation is also not available in MAS based simulators. This makes signal modulation and demodulation more difficult to analyze since it requires complex number calculation and constellation diagrams for analysis.

The large scale toolkits like RepastHPC and SWARM [35] are used extensively for high performance distributed computation. These simulators are popular in distributed simulation environment where many processes run in parallel and resources like memory are not shared. RepastHPC is used in workstations, computer clusters and even in super computers for distributed agent based simulation. General purpose desktop software like Matlab and NS-2 can also be used to model multi agent systems. They often have an extensive set of mathematical functions and add-on libraries which are very helpful while a system has a high demand for mathematical relations or functions to model MAS. However, these sets of general purpose software do not have built-in functionalities for agent based models. Thus a programmer has to build these functionalities form a scratch and which becomes a tedious task for the developer especially when designing a micro level behavior of individual agents from which a macro level pattern of an organization will emerge. A user can also develop agent based models in an integrated development environment like Eclipse [36] using any languages like Java, C, Erlang etc, but again, it would miss the agent based modeling framework.

3.2. MAS system model, design and implementation

Multi agent system provides a natural representation of a complex system which often is a consequence of a progressive interaction between the agents in a system model. Having said that, the design of such complex systems needs careful planning and implementation. Before designing a MAS specific model, one has to ponder about the needs to use MAS for a particular design. The decision making factors for using MAS as a system model could be outlined as follows:

• The significance of a model and the problems the model will address. • Answers the model will provide to help make a decision. • The benefits MAS would bring over other simulation techniques

(comparative analysis).

20

• The characteristics of the agents, the roles, behavior and capabilities. • Definition of authorities, for instance, who will be the decision maker in

the system. • The type of data to be generated, its source, medium and destination. • The environment where the agents will be placed, the impact of such

environment on agents and vice versa. • The features of such environment. Temporal evolution of the environment,

the basis of transformation. Adaptability of agents to the environment. • The data routing methods, communication strategies and protocols used for

specific environments. Mobility methods, state transitions and agents interaction mechanisms.

Building a MAS model will require a behavioral structure of the participating agents, platforms were briefly discussed in the earlier sections. Figure 5 describes the design and implementation process of MAS models.

Figure 5. MAS simulation process.

3.2.1. Agent architecture and behavior design

An agent architecture defines the modes of operation an agent can perform in its world and the relationship between its internal components. It is more like a blueprint for developing a particular agent. Building and analyzing the agent architecture depends upon the characteristic features of an agent. As mentioned in the section 2.2.1, generalizing the agent classification gives us a scope that can cover a wide range of

Agent architecture design

Agent behavior design

Agent environment design

Running the model

Testing and verification

21

agent types and their functionalities. The categorization of the agent architecture [37] exists in two folds: 1) reactive architecture 2) cognitive architecture.

In the reactive architecture, the agents do not possess the information of other agents

or its environment in advance. How the agent would behave in its world is triggered by the environment in which it would operate. The subsumption architecture [38] is the most popular among others in this domain which is a bottom-up approach forming a hierarchy of operations. The upper layer of operations such as a goal is a result of the simple operations in the lower layers. The upper layers can only be executed if the lower layer tasks are satisfied.

The cognitive architecture defines the reasoning ability of agents according to their knowledge derived from the past experiences and can utilize these knowledge for future actions. The environment (information or status) and other agents are explicitly represented by this knowledge [37]. The BDI architecture as mentioned in section 2.1 is one among this domain of architecture. The belief-desire-intention architecture defines that the belief is agent’s local knowledge, desire is the objective of the agent and intention is the executable means with which an agent can accomplish its objectives.

The behavior of an agent is determined by its perception from the environment and what actions it would perform on the environment. After the architecture of an agent is constructed, the next step would be to fill in the behavior of the agents in to the architecture.

3.2.2. Environment design

The environment can be taken as the function of an agent’s perception and its resultant actions. The input parameters that an agent receives are from the environment in which it acts. While designing an environment, it is imperative to understand what kinds of features an environment would hold. Is it a static environment whose temporal relationship with the agents remains constant or is it a dynamic environment which changes according the agents interaction and its actions? If we take an example of a mobile user transmitting to a base station while it remains inside the building, the environment is pretty much static in nature. But if the mobile user changes its location from one instance to another, its environment also changes according to its movement. This type of variation in the environment could be of continuous or discrete in nature depending on the time steps the mobile unit takes while transmitting. It is also important to note that the environment may or may not contain a single breed of agent. And if multiple breeds of agents are present, how the system parameters will change and to what extent is also notable. Let’s say a particular group of delay sensitive traffic that requires a high data rate and the other group has a store-and-forward data transmission scheme, both of these may require different data rates to be assigned. So a designer must incorporate both of these environments in the model that can also take care of the topological variations in the network. Therefore, a need to classify the properties of environment arises for the sake of environment design in MAS.

A clear classification of environment properties can be found in [39]. The classification is presented as:

22

Accessible: If all the states of the environment are available to the agent and if the agent can access them at any time, then the environment is said to be accessible. For example if a full channel state information is provided to the transmitting agent, then the agent need not worry about keeping track of signal propagation, scattering, power decay etc and can achieve a reliable transmission at higher data rates. If this information about the channel (environment) is not accessible to the transmitter, then it is said to be an inaccessible environment.

Deterministic: If the next state can be determined by the current state and the actions performed by the agent then such an environment is termed as deterministic. Episodic: In this environment, the agent’s previous experiences are divided into episodes and the succeeding episodes is irrelevant to the actions occurred in the previous episodes. Static and dynamic: If the environment changes with the passage of time then the environment is said to be dynamic otherwise it is static. Discrete and continuous: If the environment has accurate and finite states then such an environment is the discrete environment but if an environment sweeps through a range of infinite continuous values then it is a continuous environment. Keeping in mind these properties of the environment, a designer can tailor the environment model according to the needs of the system. This way the interaction between agents and the environment could be formalized, once the environment properties are set. A set of interactions between agents themselves, perceptions, information gathering or manipulation of environmental parameters can be added which are particular to the model. A typical view of the environment is presented below in Figure 6, which depicts a message routing environment between mobile nodes where the environmental entities are different breeds of the nodes themselves.

23

Figure 6. 3D view of a MAS environment

3.2.3. Running the model

After the environment model is composed and the interactions are integrated, it is time to run the model and make predictions from the results obtained. Running the model is platform specific. Some software run the model through graphical import and run features and some provide headless run through command line interfaces. The latter property is especially useful if the user wants to automate the runs in a single or clusters of machines. This also helps to invoke one model from another. A batch-run simulation collects the parameters from a specific parameter file where the starting and ending values of the model’s parameter are provided. Since a non-batch run is a manual procedure, the user himself has to start and stop the run through graphical interface and set the parameters accordingly.

3.2.4. Testing and verification

Testing and verification in MAS is a technique of checking the model to ensure that the programming inside the system model is functioning properly or not. With the help of this technique we can confirm the accuracy with which the outcome from a model has characterized the target system. Testing and verification seldom relies on any specific tools in multi agent systems, it rather is based on the evaluation data to be compared with data produced from any other simulation systems, thought to accurately characterize the target system. Nevertheless, the testing and verification can be

24

performed with some general criteria [40] which would help in determining whether the model has succeeded to represent the target system. These criteria are:

Correctness: This is a direct evaluation method to see how the goals are achieved, whether the model has represented the macro behavior of the organization found in the target system. Consistency: The model should be consistent internally and on the other commonly accepted theories as well. This gives a conclusion whether the model correlates with the conceptual framework or not. Simplicity: The model should not only bear correctness but also be simpler when compared to other similar models. Innovative: The model should have novel outcomes, i.e., it should come up with new discoveries, different results if the different data are fed to the model. Agent based modeling community has taken great strides in past decade offering many different toolkits, each with various characteristic features. With a number of options available at hand, a user new to modeling and simulation of MAS is likely to be overwhelmed with the existence of these toolkits nowadays. Some toolkits are made to be general purpose whereas some are specific to a domain. The ability to make a suitable choice of the simulator often comes with the user’s past experience on programming, the functionalities a toolkit provides, the type and complexity of system to be modeled and the level of support and documentation of the simulator available at hand. Thus, it becomes important to distinguish these simulators based on the characteristics mentioned above and others. In the next section, five agent based simulators are presented in context of use in wireless communication networks.

3.3. Some existing multi agent based simulators

This section provides an overview of existing simulation platforms that are well established for multi agent based modeling and design paradigm. Despite the number of available toolkits with numerous application areas and licenses, five of the major MAS simulators are presented with a perspective of use in wireless communication networks, all of which are general purpose and freely available toolkits.

Swarm: Swarm is a multi agent system toolkit that is designed to model and simulate the emergent collective behavior of the agents arising in a complex adaptive system. Swarm is one of the earliest of its kind, developed at Santa Fe Institute in mid 1990s. It is now maintained by a nonprofit organization called Swarm Development Group. Distributed under GNU General Public License (GNU GPL) [41], Swarm has been extensively used in scientific and commercial applications. The libraries are based on both Objective C and Java and messages can be passed in between both of the library types. Objective C was chosen perhaps because it is easy to learn and it facilitates dynamic binding, a term

25

used in object oriented languages in which the methods that are called are determined during the runtime and not the compile time which gives an added advantage to easiness in writing generic codes. However, increase in execution overheads is the issue. In communication networks, Swarm has been used in projects such as mobile communication database simulation [42] for location management and billing purposes in cellular networks and mobile ad-hoc sensor network [43] covering routing, energy control, medium access control (MAC) and application layer protocols. Assigning variable types during runtime and display features like probe which can be used to monitor and control simulation object is mostly advantageous while constructing models in Swarm.

Repast Simphony: Repast is, driven by its simplicity and versatility to integrate different programming language in the code. Codes are written in either Java or Relogo. The main advantage of this is- a user can begin to write simpler model with Relogo and later it can be extended to larger complex models using Java. Repast facilitates the user with its rich set of libraries for social network modeling, genetic algorithm, neural networks, all of which are of vital importance to today’s wireless communication network designs. Being a fully object oriented programming and modeling platform, Repast provides the user a freedom to access and edit the agent and model properties dynamically within the run time. Repast also has supports for more complex systems like geographical information systems (GIS) and it provides its own graphical data logging and graphical tools. Repast being implemented in Java, it inherits all the benefits from Java such as multithreading, platform independence, distributed computing etc. to name a few. The use of this agent based platform in wireless network paradigm has been discussed in chapter 5. Netlogo: Netlogo is a multi agent based modeling and simulation platform which was built in Scala [44]. It is an object oriented programming language for Java virtual machine (JVM) [45] and Java. Netlogo is sometimes also acknowledged as a programming language because of its unique high level procedural language which has a spirit of Logo programming language [30]. It is an open source software distributed under GNU GPL license. Netlogo has emerged as the most popular agent based simulator because of the high level of abstraction it provides making itself a very user-friendly, interactive and interesting simulation platform. Even a user with no programming background can learn it within few days of practice. Excellent documentations and a very active support community make the experience of using Netlogo even more worthwhile. Netlogo contains well structured sets of primitives and community contributed extensions that make Netlogo a dynamic simulation environment where users write and run their models while also playing with their codes at the same time. Extensions like HubNet provides distributed computation support and client-server environment for participation while headless batch-run simulation contributes to parallelism. The agent’s behavior are mostly reactive in Netlogo but there exists libraries that support the BDI architecture and message exchange via FIPA-agent communication language (FIPA-ACL) enabling cognitive features in Netlogo’s agent architecture. Netlogo is widely used in social simulation but recently it has gained much attention to other scientific disciplines as well such as telecommunication where Netlogo has been

26

used for telecommunication network management. In wireless communications, Netlogo is especially desirable to model the distributed network architecture and cognitive radio features implementation. It is one the most acceptable MAS simulators for wireless networks because Netlogo is particularly suitable to simulate complex networks that develop over time; one of such kinds is the wireless communication networks. The specific areas of wireless communications that has exploited this simulation platform are routing and resource management. But it is not only limited to those; in this thesis a radio environment model, time division multiple access (TDMA) MAC and a channel allocation model based on user priorities is also presented in Chapter 5, all simulated in Netlogo. Jade [46]: Jade is a Java based framework for designing distributed multi agent models. Jade is comprised of object oriented programming language and database and a graphical interface that supports debugging and deployment of models. Jade models could be simulated in the local machine or distributed in a cluster of machines. Jade simplifies the agent development and communication methodologies by a middleware that adheres to FIPA specifications for agent development and communication language. Jade also has application programming interface (API) for other languages such as C, Microsoft .NET and other web interfaces and it is flexible enough for deployment in lightweight operating systems such as Android. Jade as a FIPA complaint framework has its core architecture based on three main functionalities: i) The agent management system: responsible for managing the operations of agent platform. ii) Directory facilitator: Providing a yellow page for agents to know the whereabouts of other agents and services. iii) Agent communication channel: to facilitate the routing of information between agents in a local system or an external platform. Jade is seen to be used in internet applications, industrial prototypes and gaming applications. In wireless communication networks, the application of Jade is found in modeling wireless peer-to-peer network functionalities and also used for call admission simulation in distributed communication networks. MASON: Mason is a Java based discrete-event multi agent simulation platform that was designed for large batch run simulations and with special attentions to computationally demanding models. The primary advantage of MASON is the speed [47] and the batch runs in parallel machines. However, the graphical interface is not friendly as compared to other MAS simulators. Since, MASON is driven by adaptation, evolution and learning, it can also be a better choice to simulate distributed wireless networks with a large number of nodes.

27

4. COMMUNICATION STANDARDS IN MAS BASED DISTRIBUTED SIMULATIONS

Multi agent simulators are usually not equipped to handle tasks carried out with agents in huge numbers. Since parallelization is involved in MAS and resource consumption in large scale simulation is exceedingly high, a single desktop simulation environment quickly exhausts in terms of computational resources. There may also be a situation where a model has to support agent migration between nodes of a cluster and a single computer based simulation cannot represent this scenario naturally. That being so, a middleware platform for MAS is needed that can support distribution of agents and facilitate the agent management, information sharing and communication over a network in multi-threaded environment. In MAS terminology, this type of platform is termed as multi-agent middleware. They provide standard application programming interface (API) for service execution, inter-agent communication and information exchanges in a distributed simulation. However, in this chapter, the primary focus is on the communication standards between agents in distributed simulations. In this regard, two major communication standards are presented: FIPA and knowledge query and manipulation language (KQML).

4.1. Communications based on FIPA

The recommendations on development of multi agent systems have been formalized by different organizations but the widely accepted and used among those is the Foundation of Intelligent Physical Agent. FIPA is a non-profit based organization that promotes the development of agents and the interaction between them by facilitating standard specifications on agent architecture, services, tools and techniques for heterogeneous agents and agent based platforms. FIPA, however, does not provide the internal architecture of a platform. It rather focuses on the interoperability between the platforms in a network. As such, communication between the agents residing in different platforms becomes the key concentration of FIPA’s objectives.

One of the important aspects of the multi agent systems is the need to establish communication between the agents to keep the system consistent throughout the operation. FIPA has provided a communication standard of agent communications with three major components: 1) agent communication language (ACL) 2) interaction protocols 3) content language.

28

Figure 7. Components of the FIPA agent communication standard

4.1.1. FIPA-ACL

FIPA-Agent communication language is the language of interaction between the agents residing in different platforms. The language is based upon Speech Act theory and the messages passed between agents are considered as communicative acts because these messages are sent to perform certain actions based on the message type and the origin-destination pair of the agents. A typical ACL message is a sequence of byte streams in a textual form, composed of communicative act type, sender-receiver pair, content of the message, content language used and the ontology. A sample ACL message is shown below in Code I where request is a communicative act performed by one agent on another and the parameters used in the message are provided in Table 1. Code I FIPA-ACL message sample

(request :sender BSnode1 :receiver Mobnode4 :content “sense=senseSpectrumHoles” :protocol fipa-request :language sl :conversation-id 4325 :ontology channel-allocation )

Content Language

Interaction protocols

ACL

29

Table 1. Parameters used in the ACL message and their respective meanings Parameters Remarks

:sender Represents the identity of the sender agent :receiver Represents the identity of the receiver agent :content Denotes the content of the message (sense the spectrum for

holes) :protocol The protocol used for interaction. Provides the context of the

message sent to the receiver. :language Denotes the content language (encoding scheme) :conversation-id Represents the undergoing communication sequence :ontology Name of the scheme of which the action in the content is a part

of an execution process. Ontology is a common understanding of rules and terms used by the agents to limit the amount of interactions and focus more on a shared knowledge.

When an agent is ready to send the message to other agent/agents, the message is first encoded into a payload and then to a transport message. The encoding of the message is done in basically three formats: representing messages in the forms of hexadecimal bytes, strings and the popular extensible markup language (XML) format. The transport of the message is done via the message transport service (MTS) that is provided by the agent communication channel (ACC) in the FIPA communication architecture. This transport message contains the information about the sender and the receiver addresses, envelope containing time the message was sent, received, route etc for the MTS. Figure 8 depicts the FIPA-ACL message conversion from a raw message to a transport message ready to be sent.

Figure 8. Message transformation from raw to transport message

30

4.1.2. Interaction protocols

An interaction protocol is a predefined communication protocol for exchanging sequences of messages between agents to perform a task. These interaction protocols aid the agent implementation in tasks such as when an agent is trying to register itself in to the agent management system or a directory service [48]. In situations when an agent makes a query, requests for a particular service or seeks helps from other agents, the management of communication is facilitated by interaction protocols. This way an agent has a concrete path to follow irrelevant of whether or not the agent bears smartness. FIPA provides a set of these protocols [49] for establishing agent’s communication, some of which are basic protocols like FIPA-Request, FIPA-Query and some are more complex like FIPA Contract Net protocol.

The FIPA-Request protocol allows an agent to send a request to another agent for performing a particular action and in return the recipient has to perform the requested action or reply that the requested action cannot be performed or it failed to perform it even after accepting the request. The recipient agent does this by replying with messages like not- understood, refuse, agree if the request is not actually understood, refused because it could not facilitate the request or has agreed respectively. FIPA-Query is simply the protocol which is initiated if some information has to be retrieved from the receiving agent. FIPA Contract Net protocol is based on the Contract Net protocol [50] which is a load sharing protocol in MAS. This protocol maintains a contractor agent that distributes “the call for proposal” to several other participating agents. This “call for proposal” is the request of the proposal to perform a certain task. The participating agents return their proposal back to the contractor and one with the best proposal is selected. The selection is confirmed by sending the accept-proposal message to the selected agent and all other undesirable agents are sent the reject-proposal message indicating the proposal was not selected. Several other interaction protocols do exists such as variants of the above mentioned protocols and auction based protocols, all of which are interaction dependent.

4.1.3. Content language

The content language tells the receiver about the action that it is supposed to perform and how the message should be parsed. It should also be expressive enough to clarify objects, propositions (certainty of a condition) and actions (activity) [49]. The examples of content language are Knowledge Interchange Format (KIF) [51] , FIPA-Semantic Language (SL) [52] and OWL DL [53]. All of these languages describe the contents of the messages but none are general purpose languages but are rather specific to their application areas.

4.2. Knowledge query and manipulation language (KQML)

KQML is a high level interaction language and protocol based in Speech Act theory for exchanging information and knowledge (facts about a system’s domain) between platforms or agents. KQML was developed because of the need of a common language

31

for communication between knowledge based systems and only later it influenced agent based modeling community to be used as inter-agent communication language. The agent’s communication is with regard to the knowledge base, a common platform for sharing information. Regardless of the communication media adopted, may it be TCP/IP link over internet, Simple Mail Transfer Protocol (SMTP), Internet Inter-ORB Protocol (IIOP), Signalling System 7 (SS7) short message service, KQML assumes the communication to be point-to-point message passing.

In FIPA-ACL the activity of an agent to perform a certain action by the means of communication is called communicative act. In KQML this message sent and received among agents is called perfomative. A performative is expressed as ASCII string which is based on Lisp [54] syntaxes. The choice of ASCII-string Lisp is focused on the easiness for humans to read, simplicity for parsing by programs and transportable among messaging platforms [55].The message structure of KQML is similar to the FIPA-ACL because syntactically both the languages are identical but they are different in semantics. Sample code snippets showing the message structure of a communication taking place between two agents are shown below in Code II and III. A handover scenario is considered where in first case the agent eNodeB1 asks one of the user equipment (UE) to send the status report and in second case the UE2 sends the status report with request id in order to correlate the request with response. Code II KQML message structure with ask-one performative (ask-one :sender eNodeB1 :receiver UE2 :language prolog :ontology handover :content”measurement_report(handover_thresh ?status)” :reply-with request-106 ) Code II KQML message structure with reply performative (reply :sender UE2 :receiver eNodeB1 :language prolog :ontology handover :content”measurement_report(handover_thresh below)” :in-reply-to request-106 ) Conceptually, the KQML expressions could be thought as a content being encapsulated in the message wrapper which is further encapsulated in the communication wrapper

32

[56]. This way KQML can be viewed as a layered model divided into three layers as depicted in Figure 9: Content layer, Message layer and Communication layer.

Content

Message

Communication

33

5. USABILITY OF MAS IN COMMUNICATION NETWORKS AND COGNITIVE RADIO

This chapter first introduces the use of multi agent systems in radio access technologies and subsequently a simulation model for a radio environment is presented in sub-section 5.1.1. It then advances to other areas of wireless communication networks where agent based simulations can be applied.

5.1. Simulation model design for communication networks

In order to investigate the suitability of MAS based simulators to model the underlying phenomena of communication networks, three models are developed in Netlogo. The first one is the model depicting the characteristics of radio environment in section 5.2.1. The second one is a model that represents the TDMA frame structure of a GSM network in section 5.5.1 and lastly the channel allocation model based on priority is presented in section 5.6.1. All of these models are developed as a part of this thesis with an aim to manifest the feasibility of simulating communication networks phenomena with multi agent simulators and to signify its importance in modeling lower to higher layers of communication stack.

Netlogo was written in Scala and Java, so it runs on top of Java Virtual Machine (JVM) enabling itself to run on multiple platforms (for example, Linux, Windows and Mac.). It is also an object oriented programming language inspired by the Logo programming language, used to model complex phenomena evolving through the emergent behavior of the agents involved. A typical system model in Netlogo consists of independent agents that are called turtles, the environment over which these turtles move around are called world that consists of square grids of patches across which turtles exhibit their behavior and cooperate with other turtles aiming a certain goal. The objects in Netlogo are called procedures, where each procedure can be called by another inside a model. Netlogo facilitates the user by providing inheritance, double precision floating point math, network extension, directed and undirected links for node to node connections with built-in primitives which are very helpful while designing radio environment and its elements. The choice of this platform among others is based on factors like quick and easy implementation of the logics, capability to address individual nodes in the environment, real-time playing with nodes and the feature of generating statistical data for the variables used in the environment. The architecture of the simulation model developed in this thesis is shown below in Figure 10.

34

Figure 10. System architecture of the simulation model in Netlogo

5.2. Radio Access Technology (RAT)

The co-occurrence of heterogeneous networks often characterized by being large scale networks with diverse radio access technologies (Cellular, wireless local area network (WLAN), WiMax, bluetooth, Ad-Hoc, ultra wide band (UWB) etc), diverse cell structures (macrocells, microcells, picocells, femtocells) and equipments (access points, nodeBs, base transceiver system (BTS), routers, etc.) as well as different QOS and user requirements with time varying radio channels impose new challenges to the wireless networks.

Apparently, it has been realized that the current radio resource management (RRM) mechanism struggles to achieve the requirements of multi-radio access technologies. This calls for networks and terminals to bear fast and reconfigurable capabilities along with complex optimization- based techniques which would manage the resources efficiently. In [57], a novel distributed intelligent MAS has been proposed which is perceived to address the issues produced by co-existence of multi-RATs operated by different network providers. The agent performs a macro control and management

35

function by using control factors and validation mechanism instead of micro control handling detailed RRM decisions concerning individual users [57]. A self-organizing system for the beyond 3G networks have been proposed that includes a Multi-agent system architecture comprising a radio access network (RAN) and the interfaces between RAT-agent and agent-to-agent. The RAT-agent interface facilitates an agent to gather the radio network information, for instance, the parameters of radio resource, performance, etc., and sends control factors to the RAT. The agent-agent interface provides a pathway for the coordination and functions for the inter-network resource adjustments while also coordinating with the neighboring agents and verifying the adjustment results by using validation mechanism for evaluation of appropriate adjustment.

When we consider a cellular network, for example, a Global System for Mobile (GSM) network, the basic operational parameters such as carrier to noise ratio (C/N) on the edge of the cell, carrier to noise plus interference ratio (CNIR), channels per cell, Log-Normal Shadow fading, cell area probability (CAP) are also taken in to account. Furthermore, modeling the characteristics of radio channel such as RF propagation models (Longley-Rice model (Hufford G.A., 1982), Okumura model, Hata model etc), attenuation, signal to interference plus noise ratio (SINR), bit error rate (BER) for a corresponding signal to noise ratio (SNR), received signal strength indicator (RSSI) and other link budget factors cannot be left alone while modeling a holistic approach to a network may it be centralized or distributed in nature. Implementing MAS to simulate these characteristic features of wireless communication can be performed with a shallow learning curve. Propagation loss is a factor of distance, frequency, scaling constants and calculation of such an attenuation with empirical propagation models, free space path loss (FSPL) (for LOS), COST 231Hata [58], Stanford University Interim (SUI) [59] model (NLOS) can be simulated in a multi agent environment provided the parameters for calculations are known. In the literature [60], the author has presented the localization and tracking procedures based on RSSI (Received Signal Strength Indicator) signals making use of a multi agent system simulation toolkit called Repast Symphony.

5.2.1. Simulation model design for radio environment

This simulation model for path loss and received signal strength (RSS) is developed as the first model in this thesis. Factors such as energy per bit to noise power spectral density ratio (Eb/No), SNR, BER for the corresponding SNR etc. are also modeled and used in simulations. Radio propagation loss models used in this simulation are the FSPL and Hata model for urban areas. Free Space Path Loss: It is the loss of signal strength attenuation as it travels a distance through a wireless medium. FSPL is proportional to the square of the distance and also to the square of the frequency of transmission between a transmitter and a receiver. The equation defining the path loss is given as:

36

FSPL = �4𝜋𝜋𝜋𝜋𝜆𝜆 �

2 (1)

= �4𝜋𝜋𝜋𝜋𝜋𝜋𝑐𝑐

�2

The entities d is the distance between the transmitter and the receiver in meters, f is the frequency of transmission in hertz, λ is the signal wavelength in meters and c is the speed of light in vacuum, the speed of which is approx. 2.99792458 × 108 meters per second. In decibels, it is expressed as, FSPL (dB) = 20 log10(d) + 20 log10(f) + 32.45 (2) where, d and f are in kilometers and megahertz respectively. Okumura- Hata model: This model is often used to predict the behavior of cellular transmission in cities with buildings where signal propagation is affected by diffraction, reflection and scattering by the structures. This model has been extended to Urban, sub-Urban and Open areas models. The model used in this simulation is for the urban areas that accounts for the losses due to obstructions created by the building structures. The mathematical formulation for this model is given as: Lu = 69.55 + 26.16 log10 f - 13.82 log10 hB – CH + [44 – 6.55 log10 hB ] log10 d. (3) For urban areas, CH = 3.2 (log10(11.75 hM ))2 – 4.07 (4) where, Lu = the path loss for urban areas in dB. hB = height of the antenna of the base station in meters. hM = height of the mobile antenna in meters. CH = Correction factor for the antenna height. The received signal power is based on the Friis transmission equation [61]. C/N or SNR is calculated by the equation 5.

𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑃𝑃𝑠𝑠

𝑃𝑃𝑛𝑛 (5)

37

Where, Ps = power of the received signal. Pn = noise power and is given by equation 6. Pn = KTB*NF (6) Where,

K = Boltzman’s constant = 1.38 *10-23 Joules/Kelvin T = absolute temperature = 290 kelvins B = bandwidth in hertz NF = noise figure of the receiver And,

𝐸𝐸𝑏𝑏𝑆𝑆𝑜𝑜

= Ps

Pn

BR (7)

Where, Eb = energy per bit (or average modulating signal power in each

bit duration). No = Noise spectral density in dBm/hertz and R = bit rate (inverse of the bit duration)

5.2.1.1. Functionality of the model

The model describes a radio environment where the nodes are distributed randomly across 16 x 16 grid-space, the frame rate chosen is 30 frames/sec at normal speed and the base station lies in the heart of the world. The wireless nodes maintain a constant communication with the base station and at any instance the characteristics of the link can be calculated and analyzed. Functions such as path loss, BER, SNR, EIRP, Eb/No, RSS, communication range are the major elements covered by this model. The calculated values and messages are being reported in the reporter box and in the console window. Table 2 summarizes the simulation parameters for radio environment.

38

Table 2. Parameters used in radio environment simulation

Nodes Value Units Mobile Height Variable (user defined) Meters Distance from BS Variable (user defined) km Mobility Randomly variable Time Steps Base Station Height Variable (user defined) Meters Tx_frequency 990 (user defined) MHz Receiver Bandwidth 5 (user defined) MHz Tx_power 30(user defined) Milliwatts Number_nodes User defined Nodes Data Rate 512 Kbps Communication_range 12 (user defined) km Frame rate_model 30 Fps

Global Value Units Base Station 1 Node The data calculated from Netlogo can also be exported to a data file such as csv, xls, xml etc and can be used to print a nice graphics in other simulators like R, Matlab etc. The following snapshot Figure 11 of the current model depicts the radio environment and the subsequent Figure 11 shows the plot of “Eb/No vs. BER” in Matlab from the data generated by Netlogo.

39

Figure 11. Netlogo simulation model for radio environment

40

Figure 12. Exporting data from Netlogo to Matlab.

5.2.1.2. Mobility in the model

The radio environment model has an added property that the nodes exhibits mobility based on a random walk. Each time a procedure is called for heading towards random direction, the primitive, random-float generates a floating point number between 0 and 360 degrees. According to this value, the nodes sets its orientation and move a step forward based on the value of transition provided by the user. While the nodes are moving, they also continuously maintain communication with the base station and the values obtained from the calculation results due to varying parameters are seamlessly plotted each time the nodes move within the world. The model also adds the functionality of out of coverage zone. Whenever the node of interest travels out of the specified communication range, the base station detects the node as “Out of Range!” and the procedure for calculation halts, exits and resumes back to normal on next iteration cycle.

This model illustrates the modeling of basic functions like pathloss and BER calculations which are essential in simulating radio propagation and channel modeling. By using a multi agent based simulator, these functions can be easily modeled while monitoring the evolution of system and results obtained through even during the runtime. Netlogo and other simulators like Repast S also provides GIS support, which enables a user to import vector and raster GIS data in to the model and work with them.

41

In Netlogo, a GIS integration could be achieved by first performing transformation between the GIS dataspace and Netlogo dataspace and then loading the vector or raster datasets into the model. This could be beneficial if a user wants to perform radio planning in a certain geographical region. With a high level graphics rendering features, multi agent simulators like Netlogo could be a valuable tool to model future wireless communications with ease and simplicity. However, the tick based time slicing used in these simulators is not appropriate for simulating delay sensitive models. This imposes a difficulty to model communication channels where multipath propagation is involved.

5.3. Resource management and load balancing in wireless networks

In wireless communication systems, the requirements of “always on” and reliable often fulfilling the required grade of quality of service (QoS) creates complex optimization problems. Interference, power and channel allocation in the face of time-varying frequency/time-selective radio channels are not uncommon matters of concern. Moreover, with the mobile communication networks being more heterogeneous in nature, data rate requirements and latency vary from user to user and device to devices. The extensive use of multimedia and cloud services in high speed communications systems demands to design the systems capable of high speed data transfers and at the same time, fairness to all the users. This is achieved by the virtue of efficient resource allocation schemes in wireless communication systems. With an efficient power allocation and rate assignment strategies, these physical and link layer resources can efficaciously be managed.

Simulations with MAS are suitable in distributed systems because of its inherent nature of being scalable and autonomous, not to mention the cross-platform language it can be written in (for example Java). User mobility is yet another problem as frequent handovers occur with high mobility speeds and resource allocation becomes more complex. The bandwidth allocation in the modern wireless system will be done dynamically to the users. One such important aspect of resource management is the call admission procedure including call blocking and call dropping operations. This is performed in the cell area of the call being admitted as well as by taking in to considerations; the neighboring clusters. In [62], a multi agent system for call admission resource management for wireless multimedia networks has been proposed which addresses the drop offs due to handover for a new user by reducing the call dropping probability. This is a participation motivated system that allows the neighboring cells to participate in the decision making of call admission to a new user while maintaining high resource utilization. Thus, each of the cells or base stations has agents running with them. A basic representation of this system is shown below in Figure 13.

42

Response, R(x)

Agent

Agent Agent

Mobile Switching Center

Mobile Switching Center

Figure 13. Multi agent systems in wireless network Each agent running on the cell where the admission request has been made, will gather the information regarding the available resources from the neighboring cells and makes its own decision based on a particular algorithm to whether or not the request is accepted or denied. This increases the probability of success that the new request will survive

43

after a handoff. The final decision of the call admission process for a particular user 𝑥𝑥 is made on the basis of the equation (8) found in [63] as, 𝐷𝐷(𝑥𝑥) = � �𝑆𝑆𝑘𝑘(𝑥𝑥 ) X 𝑊𝑊𝑘𝑘(𝑥𝑥 )�∞

𝑘𝑘𝑘𝑘𝑘𝑘 (8)

It is assumed that each of the response,𝑆𝑆𝑘𝑘(𝑥𝑥 ), from the agent 𝑘𝑘 is a percentage between 0 % (completely unsupportable) and 100 % (supportable). The responses are based on the dynamic mobile probability (DMP), the probability that one user is active in another neighboring cell. By using above presented equation this response is in turn summed with the weights 𝑊𝑊𝑘𝑘(𝑥𝑥 ) to form the decision [62]. The flow chart for the admission decision of the call request in [63] is represented as below in Figure 14 where the area shaded by green is the procedure taken by agents from the neighboring clusters, all other process except that is performed by the agent in the cell where the request has been made.

44

Can call be supported locally?

Send DMPs to agents in K(%)

Receive DMPs from other agents

Can call be supported

locally?

D(x) > acceptance threshold?

Call Admission Rejected

Call Admission Accepted

Call Admission

request

Can user be supported from its arrival till it leaves?

Compute Rk (x,t)

Compute Rk(x)

Send Rk(x) to the corresponding agents

Receive Rk(x) from agents in K(x)

Compute D(x)

Yes

No

No

No

Yes

An agent in the cluster

Agent receiving request

Figure 14. Flow chart of the call admission procedure The simulation has been performed in this multi agent platform where the results show that the implementation of multi agents systems enables the wireless network to lower the call dropping probability and offers high average bandwidth utilization. In [64], the authors suggests the benefits of MAS specially of the cooperative and negotiation based qualities for maintaining QoS in cellular networks with call admission control strategies in different traffic conditions. The proposed model utilizes the concept of SHUFFLE [65], The European Commission’s Information Society Technologies project introducing agents as a control and management entities for business interactions and radio resources. The SHUFFLE model describes the functional architecture as facilities plane, negotiation plane and resources plane. This model is used by the authors to define and implement agent based call admission schemes, service architecture and dynamic

45

channel borrowing schemes. The work is implemented in multi agent based simulation framework called Jade. Jade is a multi agent system framework [66] fully compliant with the FIPA that provides a standard of implementation for agent communication language FIPA-ACL enabling communication between the agents. Jade uses internal message transport protocols (IMTP), event passing and remote method invocation (RMI) for message delivery according to the platform and locations of the containers [67] (components that hold agents) thus passing messages asynchronously between agents. The authors of the paper mentioned above have introduced a new plane called Connection Plane to the SHUFFLE architecture and also introduced an agent named Network Provider Cell Agent as a hybrid agent capable of interacting with other agents in the system. The two multi agent based service architectures, distributed and centralized were compared and the evaluations were performed on the basis of reactivity, traffic load, scalability, handoff and call blocking probability. This paper thus addresses the application of multi agent systems and focuses its benefits on call admission control and channel borrowing strategies. With the utilization of agents on decisions making for call admission and handoffs, a cooperative and autonomous network could be designed and the simulation of this perspective can be efficiently performed in the MAS framework like Jade, a powerful Java based multi agent simulation platform with matured libraries.

5.4. Applicability of routing algorithms with MAS

Use of agent oriented programming facilitates the individual components of the system by granting reactiveness, which in turn tends to improve the general efficiency and reactiveness of the system as a whole. In the scenarios where the resources are potentially limited, multi agent systems components must maintain the interactivity through coordination and cooperation. One such area of wireless communication is the wireless sensor networks (WSNs). The nodes in such networks are wirelessly connected and resources like power are of crucial importance. Having said that, the communication between nodes in wireless sensor networks is mostly broadcast based and done in a multi-hop fashion where every node that receives the data forwards it to the neighboring node closer to the base station. Since the agents in the MAS are goal oriented possessing information about the environment and the experiences from the past, coordination as the main form of interaction and decentralization as the common form of system model, they are better for the simulation of wireless sensor networks which tend to be resource efficient especially in terms of power, storage and processing cycles.

In [68], the author has discussed the potential of using multi agents in wireless sensor networks by development of SAMSON [68] simulator. SAMSON is a multi agent simulation model designed with Agentspeak [69] based interpreter JASON. AgentSpeak is a BDI (Belief Desire Intention) based agent oriented programming language and in JASON platform, an extension of AgentSpeak programming language is used for development of multi agent systems. The choice of this platform was based on the characteristics of the platform as it is object-oriented, has inbuilt agent reasoning engine, logical and agent constructs supported, has built-in agent communications medium and distributed MAS support is available and respects FIPA specifications of standard. The

46

work was tested with the use of flooding protocol for routing and a server management scenario which is an agent implemented version of the existing ArsLogica project called STATUS [68], [70]. The work has presented the simulator that is general purpose as well as it assimilates the task of purposeful research on agent based wireless sensor networks. A screenshot of the SAMSON environment model has been presented below in Figure 15.

Figure 15. Sensor Network World view of SAMSON showing nodes and random

obstacles (borrowed from[68]) Flooding, however is not an efficient data forwarding mechanism in wireless sensor networks as the network life time is the major concern for a designer and due to the fact that a single dead node can partition this kind of network to a large scale. Due to the repetition of messages broadcasted, the energy consumption in nodes is higher, ultimately leading the node to drop out of the communication link. Nevertheless, flooding as a data dissemination technique can be conceived as a basic system model that be implemented in multi agent systems to reflect the how the sensor nodes will behave as they would in similar physical scenario. One method of minimizing data redundancy in WSNs is the data aggregation technique. This protocol is specially suited for cluster based environment where elimination of redundant data transmission is aimed

47

whereby increasing the lifetime of the energy constrained wireless network. Each node in the network send the data to the cluster-head, the cluster-head then aggregates the data and sends to the corresponding base station. In [71], the authors have proposed a routing technique for energy efficient communication mechanism based in density based spatial clustering of application with noise (DBSCAN) algorithm. Making the use of multiple agents representing the WSNs, simulation based experiment performed in Repast Suite, a multi agent based modeling and simulation platform. Direct diffusion and DBSCAN strategies were both simulated with and without data fusion, showing the remarkable improvement over the data diffusion routing methodology in the network layer. It can be seen from the simulation results that the data aggression has indeed proven to be an energy aware method in which messages are routed to the base station via the route of cluster-head to cluster-head. One of the Repast versions named Repast High Performance Computing (HPC) typically has been developed for works to be performed in a parallel distribution fashion. A computer cluster networks can be a good example of this type of computing and simulation. One of the core libraries used in HPC is the Message Passing Interface (MPI) which is the directory of C++ programs that makes parallel operation of HPC possible. A simple example of the MPI script that a user would write which runs on multiple PC can be written as in Code IV: Code IV A sample script of message passing interface (local) if ( this is processor A ) then call a method from class X else if ( this is processor B ) then multiply a matrix times a vector end However, a data stored in one PC could be completely different from another PC in which case a user would write the code as in (V) to call a library function from transferring the data from one PC to another.

Code V A sample script message passing interface for distributed operation if ( this is processor A ) then call function_Send ( X ) //send the data X else if ( this is processor B ) then call function_Recv ( X ) //receive the data X end

48 Packet forwarding using the shortest path is also an efficient approach in energy constrained networks. Using intermediate neighbor nodes with the shortest and good link capacity, these nodes can be used as relays to forward packets from source to the sink thereby saving considerable amount of transmitting power. Network projection in Repast Simphony is a method of establishing abstract or physical relationships between agents with the help of tools to construct networks or graphs. A class provided by Repast for the network creation is the NetworkBuilder class which allows the user to customize the network to a specific need. An example package to find the shortest path between the source node and any other node in the network is the repast.simphony.space.graph.ShortestPath<T> based on the Dijkstra’s shortest path algorithm. A similar approach on multi-hop routing scheme has been provided with the code in [25]. It could be seen that this agent based model aims to examine the adaptive nature of the agents in an energy saving perspective within a decentralized network. The wireless nodes are scattered along the static environment with variables like energy, geolocation and channel estimation. The source and the destination nodes initialize the communication and data transmission by sending out request to send (RTS) and clear to send (CTS) packets. The source nodes calculate the path distance and link quality of the nearest or second to nearest node and select that as the relay node to forward the packet. This process continues until the packet has been successfully routed to the destination node. In each time step, the energy spent during transmission is measured and it can be observed that the energy spent using the shortest relay route is comparatively lower than that of direct communication route. The code for this model is freely available at Netlogo community model website. A quick screenshot is provided below in Figure 16 displaying the modeling environment in Netlogo. It is therefore imperative to note that the adaptive behavior amongst the agents can be easily analyzed using this type of model which is programmed in Netlogo [26].

49

Figure 16. Wireless sensor network model (Extracted from the code provided by [72] ) A similar routing strategy is the on-demand routing protocol which is a reactive protocol for route establishment in ad-hoc networks only when it is needed. This routing approach advocates the lesser overheads for routing as route discovery and establishment is the function of demand for power conservation unlike proactive approach where routes are established regardless to the need. The major disadvantage of this approach is high latency while finding the route because the route request packet is being flooded to all the neighbor nodes in the network. Also, the energy consumption used in route discovery is higher. Several modifications to this protocol have been suggested, for instance using cost metrics based on remaining battery levels in each node [73]. The main idea of this technique is to route around the nodes with low battery level and reroute if the current node run low in energy so as to survive the network from being partitioned. Another such routing mechanism based on on-demand approach has been proposed in [74] which is a simple Pheromone (attractiveness) based on-demand routing protocol for sensor networks. This data centric protocol is motivated from the idea of ant-based routing, where each ant excretes a chemical called Pheromone which triggers a social response between other ants in that particular colony. The ants following the path with highest concentration of the chemical and overall intelligence of these small creatures to achieve a common goal is much analogous to the distributed communication

50

network agents aiming to perform a common task. Pheromone, i.e., the link cost is calculated based on the energy levels, signal strength and the buffer memory of the nodes. As synchronization is required for listen/sleep scheme, this is performed using the frame synchronization method where nodes transmit and receive SYNC packets among the neighboring nodes using an assigned frame schedule. This can be thought of a medium access strategy for data transmission even though MAC functionality is not explicitly defined in the model. The modeling and simulation is carried out in the Netlogo multi agent simulator.

Thus, the functionality of independent agent adding up together to form a network that is both adaptable and scalable is a huge contribution in simulation of distributed communication networks. Routing is one such scope where agent based simulation is a perfect fit where cooperation is the most needful element to make the network intelligent.

5.5. Modeling Medium Access protocols

In cellular systems, the geographic area is divided into segments called cells. By doing so, a set of frequency in a limited bandwidth can be reused in another cell and hence the capacity of the overall network is maximized when compared to a single transmitter technique in a particular geographic area. Now, whenever a common media is shared by multiple users, it is often desirable that the users should be able to send and receive messages simultaneously while maintaining a sufficient level of QoS. Among the major channel access techniques in wireless communications, a time dependent medium access technique, time division multiple access (TDMA) is discussed herewith.

TDMA enables the users to use the common channel for transmission and reception but allocates a unique time slots to each user within a frame in the downstream and upstream thereby separating the users over the time. In a medium access technique, inter-user interference is inevitable but TDMA approach is employed in such a way that the inter-user interference is prevented by strictly adhering to the assigned slot schedules and guard time (as guard intervals) between the users [75] . The GSM frame structure is composed of multi-frames (traffic & control), super-frames and hyper-frames all of which is the groups of building blocks called frames. These frames are the defined division of the digital information consisting voice, data and other control signals. The information is communicated using prescheduled time slots grouped together consisting 8 time slots (0 to 7) per frame. The frame itself is 4.615 ms long allocating each time slots of 0.577 ms or 156.25 bit duration in which the data burst is slightly shorter than the time slot, i.e., 148 out of the 156.25 bit periods and the rest (8.25 bit) as guard bits for time dispersion and other alignment errors [76]. TDMA MAC based protocols are an important channel access mechanism in wireless ad-hoc, sensor and cognitive radio networks.

51

5.5.1. Simulation model for medium access

A model of a typical TDMA frame structure for GSM network in Figure 17 has been developed in Netlogo as the second model for this thesis. This is a very basic representation of the single frame where each slot is used for an uplink (from a mobile phone to the base station) data transfer and also depicting slot assignment to each user in a round-robin fashion with a slot duration of 577 μs and a frame duration totaling to 4.61538 ms. For each slot duration only one user is allowed to transmit and the calculation for SNR is performed calling a procedure at the beginning of the slot. A sample script of this model is shown below in Code VI :

Code VI A sample script of TDMA frame model set Slot 0.577; while[i < num-nodes AND framedur <= 4.61538] [ crt 1 [setxy 1 2 forward 10 setxy random xcor random ycor set shape "person" set color 52] reset-timer; while[timer <= Slot] [ print word(" CALCULATION FOR NODE ")(i + 1) ask turtle ( i + 1 ) [create-link-with node 0] ask links [set color blue] let a [xcor] of turtle i + 1 let b [ycor] of turtle i + 1 let hyp sqrt(a ^ 2 + b ^ 2) set Dist hyp * 1000; calculate ] ask links with [color = blue] [die]; set i i + 1; set framedur framedur + timer; print word(" FRAME DURATION ")framedur; ] end

52

BA

CD

A1 B1

C1D1Freq

uency

Time

0 1 2 43 5 6 7

TailUser Data User DataTraining SequenceTail S S Guard

3 bits 57 bits 1 26 bits 1 57 bits 3 bits 8.25 bits

4.615 ms

0.577 ms

Burst of 0.546 ms

A TDMA Frame

Figure 17. A TDMA frame showing a normal burst. Now when distributed control is the requirement such as in wireless ad-hoc networks, all the participating nodes in the communication system are involved directly or indirectly for medium access without the need of centralized management and control. A carrier sense multiple access (CSMA) protocol requires the nodes to listen if the transmission is ongoing and if a collision has occurred and if it has, the node waits for a period of time and senses the medium again. So it is a stop-and-wait procedure for medium access. With the use of RTS/CTS hand shaking mechanism (as in CSMA/CA), the probability of collision could be reduced. However, the overhead is too big for small packet transmissions due to high level of noise in wireless ad-hoc environment. Furthermore, the CSMA mechanisms are highly prone to the hidden node problems [77] which occur when the data transmitted from one node goes completely undetected from another node which is not in the range of the transmitting node. With collision detection and handling, the CSMA based protocols effectively lowers the precious bandwidth and may increase latency potentially in wireless networks. So a natural choice of medium access control would be a distributed TDMA approach. Space-time division multiple access (STDMA) [78], a variant of the TDMA protocol, is a position based TDMA scheduling protocol specially for mobile Ad-Hoc network (MANET) which is designed for spatial reuse and time slots are assigned to space slots that are evenly grouped repeating frames. The authors in [79] have shown that it is a self scheduling protocol that is tough enough to recover the network from noise and low mobility conditions in wireless sensor networks. STDMA based medium access method is used for slot allocation featuring a self-synchronization approach. The slots are joined together to form a frame. These frames are grouped according to the positions as collection, firing and checking frames which periodically repeat itself at the end of each operating cycle. Each packet consist of 6

53

bytes of a known preamble sequence, 4 bytes of management fields, 16 bytes of payload and 4 bytes for cyclic redundancy check with a time slot of 50 ms. Time slot numbered 0 is used for beacon transmission in the network. This way, the packets are received for forwarding at the frame collection-position; presence of transmission is detected at checking-position and packet scheduling at firing-position. This TDMA based MAC and scheduling has been modeled in Netlogo as a part of the whole proposed algorithm in [79]. Netlogo provides a very interactive environment to improve and test algorithms simultaneously, thus reducing the debugging practice to a great deal while a complex system is being designed. Because this simulator has been helping researchers of social phenomena by representing the self-organization naturally, it could be one of the best choices of simulation platform especially when a system requires automation capabilities to be integrated while modeling.

In communication networks, designing medium access protocol is one of the important tasks for engineers because the participating nodes in a network contend for channel access and conflicts between the nodes are likely to occur during packet transmission. This arises a serious drop in bandwidth utilization and overall throughput of the network. Therefore, a proper collision detection mechanism will become necessary to displace the nodes temporally with suitable backoff algorithms. In case of carrier sense multiple access with collision detection (CSMA/CD), the binary exponential backoff algorithm is commonly used for packet retransmission after the collision has been detected. These types of algorithms are suitable in wired network but in the case of delay sensitive wireless scenario, the backoff latency due to contention window size during collision is of major concern. Furthermore, bandwidth efficient collision detection is difficult to achieve due to the dynamic nature of attenuation in wireless environments. So, collision detection mechanism is not favorable for wireless network which is superseded by collision avoidance instead. But energy consumption still remains another hurdle in wireless cooperative networks where the burden of network management is distributed among the participating nodes. Although, this way, a load is balanced in an overall network. So, for optimizing the energy saving methods through cooperation requires a rigorous study and modeling of the MAC. While modeling the cooperative networks in multi agent simulators, a statistical approach to channel contention could be realized. For instance, if a random variable X that stores the average number of collisions over a certain time then according to Poisson’s distribution, the probability of occurrence of collisions or the probability distribution function can be calculated and according to this value, the slot assignment frame for the nodes could be determined. The Poisson’s probability density function (PDF) is given by,

f(x | λ) = 𝜆𝜆𝑥𝑥

𝑥𝑥!𝑒𝑒−λ (6)

where, x is the number of collisions that can take random values and λ is both the mean and variance of distribution.

In Netlogo, a primitive called random-poisson can serve a better deal while modeling this scenario. This is only an example of a probabilistic approach for modeling a part of the MAC for cooperative networks with multi agent simulators. Another approach could

54

be first-come-first serve basis scheme in which a particular node which is most active within a group (based upon its social characteristics) could be granted the authority to assign time slots to the other nodes for packet transmission. A similar approach can be found in the work of [80] which uses a game theoretic method to model contention between nodes in a cooperative way. The authors propose an energy efficient MAC scheme that uses a TDMA like methodology for medium access between agents. In the first case scenario, the performance of a cooperative system in an ideal case (no collision) is evaluated and compared with the system using state of art IEEE 802.11 [81] \WLAN. In the second case, three of the scenarios: Ideal, WLAN and proposed scheme are compared using a game theory approach. The cooperative strategies used between players are the wise and selfish where the wise represents the strategy in which players tend to minimize and conserve their energy whenever the energy consumption seems to be high. Whereas in the wise strategy the players have knowledge of power consumption of the cluster as a whole and use this as a factor for making decisions in cooperation. CSMA/CA is used as a contention method for choosing the cluster head which is responsible to assign time slots to all the nodes in its cluster so that they can use it for data transmission. This can be done by sending pilot tone to the members of the cluster and each mobile node receiving this sends its sequence of substream to all mobile nodes in the cluster. Game theory seems to be a suitable method for this model as game theory provides a fundamental representation of a real cooperative network in which decision made by each player affects other players and thus a collective impact to the overall system. This is yet another illustration of a MAC layer implementation in wireless ad-hoc systems using multi agent system and toolkit, Netlogo. The beauty of multi agent simulator lies in its simplicity of coding methods which is complimented by the great abstraction level of these simulators. As an example scenario, a Unix system most of the input/output operations are considered to be a stream of bytes written or read to/from a device which actually can be a model to open a port or a socket. The programmer, however, only uses these bytes whereas the operating system provides an abstractive interface by dealing with the device’s physical characteristics. Similarly, the multi agent simulators like Netlogo as general simulator software, provides a high level of abstraction to the programmer that enables him to focus on the logic instead of the brainstorming about underlying complexity of the system. This feature of the MAS simulator could be an added advantage to model complex phenomena of wireless cooperative systems such as MAC. Thus, these simulators are evidently a better choice to model a time-bounded packet transmission like TDMA MAC that can provide essential information for routing such as node discovery, determining link metrics and efficient response to change in a topology.

55

5.6. Modeling Cognitive features

The demand of future wireless communications to provide a seamless and ubiquitous connection to the mobile users in a heterogeneous network environment has led to profound research and development of intelligent radios. These intelligent radios should make the best possible utilization of the available spectrum and should guarantee an acceptable level of interference among the primary and secondary users of the subbands. The inefficient use or the underutilization of the natural resources such as bandwidth is the consequence of static allocation in a communication network. The utilization of bandwidth at any given time is a stochastic phenomena and analysis of such an issue is merely a subject of probabilistic scrutiny. Moreover, detecting the spectrum hole [82] in an opportunistic manner for cognitive purposes is yet another challenge as it is also nondeterministic in nature. So the solution to these issues lies in the development of such a network that can sense and learn the situations, smartly and dynamically allocate the radio resources and should be cooperation motivated. Such a notion of an artificial intelligence has been brought about by the concepts of cognitive radio which stands in the base of software radio platform [83]. Cognitive radio concept was first presented in literature by J. Mitola III in 2000 [84]. The author also defines the cognitive radio as a goal-driven framework where the radio, autonomously observes, concludes the context, learns, plans and supervises the radio environment [85]. Several theoretical frameworks currently exists for the study of cognitive features such as; Lagrangian optimization techniques, game theory, auctioning, graph theory/ graph coloring; all of which can be used in centralized, distributed and cooperative communications not only in spectrum management but also in other domains like time, power, coding etc [86]. If we see the cognitive radio concept with a MAS perspective, we will soon realize the adoptability of MAS to the cognitive radio network (CRN). This statement holds true because of the fact that agents bear the qualities of being autonomous, learning attitude, reactive and most importantly the cooperative properties, all of which resembles the characteristics that builds up a cognitive radio network. Evidently, these qualities of MAS have been used in telecommunication networks study and they are still evolving. In the context of spectrum utilization in CRN, researchers have employed MAS concept to demonstrate efficient utilization of spectrum in the CRN. A survey and classification of these works are found in [87] where the existing cognitive radio proposals based on MAS are categorized into negotiation, learning and cooperation based approaches. Negotiation: Negotiation is the process in which agents converge to a particular deal or rules enabling them to reach a consensus regarding a goal or a plan. In cognitive radio, the negotiation can be applied in areas like spectrum handover. The protocols used for negotiation basically govern the interaction between the agents. Some of these protocols found in the literature are auction [88], argumentation[89] and heuristics [90]. Learning: Learning is the ability of agents to acquire new knowledge through sensing the environment (for example sensing the spectrum) and to represent this acquired knowledge in the upcoming decision making process. Learning based protocols adopted

56

in the existing works are reciprocal learning [91], Q-learning [92] and reinforcement learning [93]. Cooperation: Cooperation is the phenomena by virtue of which the agents are able to accomplice a particular goal in a collaborative manner. Without the collaboration, agents would not be able to achieve that goal on their own. This way a whole network of agents can solve a global problem. An example of this can be a dynamic and opportunistic allocation of spectrum between the nodes in a CRN. Protocols like contract net [94] and coalition formation [95] are frequently used in a CRN.

This topic, however, only focuses on the use of MAS in a cooperative network so that we could estimate the usefulness of MAS and MAS toolkits to model cognitive features in a network. In [96], the authors have proposed a distributed consensus algorithm to address the problem of existing signaling layer, common control channel (CCC) that lies in between the neighboring secondary users (SUs). The evaluation criteria of the proposed algorithm are based on the network capacity and spectrum efficiency of the cognitive radio ad-hoc networks (CRAHNs). The proposed algorithm was implemented in Netlogo and its has been shown that the spectrum efficiency of the common control channel increases with the use of the consensus algorithm and self-organized cognitive radio clusters can be achieved thereby signifying the spatial reusability of spectrum resources in cognitive radio ad-hoc networks. The spectrum efficiency improvement is obtained as a function of spectrum efficiency of consensus algorithm and random channel hopping scheme.

With self-organization being one of the optimizing techniques for topology change in dynamic mobile networks, cooperation between those mobile nodes is another important binding mechanism that allows the participating nodes (in our case, the SUs) to reach a common objective, may it be an efficient resource management problem or just a topology control method in CRNs. Cooperation is built upon a collection of individual decisions where each decision has a positive, negative or a neutral effect to consolidate a network’s overall goal. The multi agent system simulator, in this context, can be seen as a tool to predict the payoff a cooperative system can offer. In the book [97], the authors have emphasized the use of Netlogo to model cooperation in wireless networks and to analyze a decentralized wireless network from a game theoretic approach. A detailed procedure of programming the cooperative model has also been provided with clear examples. A very useful insight on the procedures to setup an environment, link establishment and control, cluster formation, how subroutines are built and called inside the program, implementation of game strategy and finally the analysis of the overall system performance on power saving and display of data are also presented in a step-by-step process. So this model exemplifies the potential of a multi agent simulator to develop such a complex phenomena of distributed wireless network that is infrastructure less and study it from a behavioral standpoint.

The authors in [98] propose a framework built on Repast Simphony that gives a clear picture of autonomous behavior shown by the agents in a spectrum occupation model. This model depicts the transition of the nodes from one radio to another according to the increase in the number of nodes. The model also presents the effects of parameter changes on the mutual interaction between the agents. Although, network simulators

57

like general purpose (OPNET [99], NS-2 [2], with added wireless components) and wireless specific (JSIM [1], ATEMU [100]) exist, the impracticality of these simulators to experiment complex scenario as that of cognitive radio is of two folds [98]: • The lack of means for simulating physical environment. • No proper mechanism to integrate spectrum occupancy data in to the model. And even if they exist [101], [102], with addressing these issues they are not tailored to suit the needs of cognitive models. Thus the multi agent based simulators come in to play as they provide a perfect playground for utilizing strategies with physical environment data, analysis of node interaction in real-time, testing the impact of variable parameters such as power, frequency, node-numbers etc in parallel execution scenarios. So, the question of whether or not it is easy to simulate cognitive radios in the multi agent based systems (MABS) platform, will not be of much importance in present context but its viability and dynamics will be.

5.6.1. Simulation model for slot assignment based on priority

This is the third model developed for this thesis which demonstrates the decision making capabilities of the central authority (in our case, the base station). The base station assigns the nodes with slots to transmit data packets based on the priority being assigned. The priority is assigned on the basis of type of data the nodes wants to transmit. Real-time voice, video and store-and-forward type of communication are considered in this model where real-time voice is assigned the highest priority and is allowed to make transmission first while the other two waits until the voice communication is completed. The number of channels available is one less than the number of nodes for each priority. The data rates for three of these communication types are also assigned on the fly according to the type of data nodes want to send. The flowchart in Figure 18 demonstrates the architecture and Figure 19 shows the simulation environment of the model. Netlogo provides primitives like list which can be used to store the data retrieved from a calculation and variables which can buffer the data during the runtime which comes handy while scanning the values in the list and perform calculations on it. With Netlogo it is easy to implement these types of model which can be integrated further to any model and perform the needful calculations.

58

Setup

Nodes request for

data transmission

Types of data (voice,MM,

store-forward)

BS prepares for priority settings

Set the priorities for

nodes

Is the data voice? Assign data

rate 96Kbps +Create link and

send

Is the data real-time video?

Wait until links are free.

Free the links.

Assign data rate 512Kbps +

Create link and send

Assign data rate 64Kbps +

Create link and send

Wait until links are free.

No

Yes

YesNo

Figure 18. Flow chart of the operations in dynamic slot assignments.

59

Figure 19. Slot allocation based on priority for real-time video data

60

6. SUMMARY The objective of this thesis is to study the feasibility of using multi agent system simulators to model and simulate wireless communication networks. In particular, the suitability of modeling physical layer, MAC layer and network layer are tested by constructing simulation models and through literature reviews.

In Chapter 2 introduction to MAS is presented and key concepts are elaborated. Application areas of MAS systems and the advantages and limitations are discussed. Chapter 3 details the model, design and implementation of MAS and presents some existing MAS simulators and their use cases in wireless communication networks paradigm. In Chapter 4, the communication standards in MAS based simulators are discussed. In Chapter 5, the use of MAS simulators in wireless communication networks is specifically presented. In initial topic the usage scenario in radio access technology is presented and along with it, a simulation model for radio environment is constructed in Netlogo. The model includes calculations for RSS, FSPL, Eb/No and BER. The model also justifies the easiness of simulating these functions of wireless paradigm in multi agent based simulators. The possible usage scenarios for example GIS integration for radio planning are also briefly discussed.

In the next topic call admission process is mainly focused in the view of previously published research works and it is outlined that using MAS, the performance in call admission and channel borrowing in cellular systems can be improved. Modeling routing mechanisms in wireless sensor networks is discussed using flooding, DBSCAN and ODD routing protocols in multi agent simulators in the next topic. It is emphasized that using multi agent simulators, routing could be made intelligent when coordinating nodes is employed in a network. A simulation model for TDMA MAC is constructed in Netlogo which represents the time slot assignments to the mobile users in a GSM network. Furthermore, cooperative power allocation scheme in distributed wireless networks with a game theoretical approach is also discussed in the view of multi agent simulators. In the last topic of this chapter, modeling cognitive radio features are explored. An overview of the existing research work is presented where autonomous nature of the nodes is studied in spectrum occupancy model constructed in Repast S. In addition, a simulation model for dynamic slot assignment based on priorities is constructed in Netlogo. This model illustrates channel provisioning to the requesting nodes by the base station according to the priority of data transmission without using any channel reservation protocols. The aim of this model is to provide an example scenario of usability of multi agent based simulators to model slot assignment in a dynamic fashion.

From the literature review and simulation models it can be seen that multi agent based simulators does provide an opportunity to create both centralized and distributed networks with ease and simplicity. However, they represent decentralized networks more naturally because of the fact that MAS based simulators are suitable to model networks with emergent complexity which involves discrete and decentralized decision makers.

From a programmer’s point of view one can argue that, since MAS based simulators are not designed for communication networks, the necessary functions to build a

61

network model are not supplied by default which ultimately leads to a lengthier code for even a simple model. Nevertheless, if build time is a secondary option, simplicity of designing communication models provides an added advantage too. Furthermore, once a model is built and verified, it could be reused while designing more complex networks. Thus, reusability is another attractive feature of MAS based simulators. It was also noted that the simulation time for running models that are scalable is insignificantly different when number of nodes are increased by large multiples. This off course depends on the complexity of the model. However, if computational resources are the constraints, MAS also offers distributed simulations possibilities to tackle these obstacles as discussed in Chapter 4. Table 3 summarizes the pros and cons of using MAS based simulators.

Table 3. Advantages and limitations of using MAS based modeling and simulation Factors Advantages Limitations

Target system representation

Better predictions can be made over models that require self-organization techniques, and to engineer emergence necessary to predict phenomena such as:

• Unexpected traffic congestion

• Uncontrollable increase in network overhead costs

• Jamming in networks when it expands to large scale.

No concrete validation mechanism available, have to depend on other established network simulators or empirical results from real experiments.

Performance Parallelism. Since multiple breeds of nodes could be created and executed, each with different attributes, multiple instances can be performed in a single model. This is very advantageous in communication systems were load balancing is the major trait.

Parallelism can impose serious impediments in execution speed. A high performance machine or distributed platform is usually preferred for MAS modeling.

System analysis

Stochastic analysis of interference and node mobility can be facilitated by extensive sets of statistical & probabilistic primitives and libraries.

No support for complex numbers. Consequently, in-phase & quadrature representation and constellation diagrams of modulated data are not possible.

Visualization Integrated plotting mechanisms and 3D rendering is available in most of the MAS simulators.

No features for logarithmic and polar plots.

62

Through this thesis, it can be implied that the usability of MAS in wireless network simulations is an important analysis on one hand and on the other hand it is equally important to realize that MAS simulators in today’s and future wireless networks are actually a need because communication networks are taking big leaps towards cognition, self-organization and cooperation, which can only be reflected effectively by multi agent based simulations.

System dynamics modeling

Channel modeling like AWGN is easier with extensions for example matrix and arrays.

Rayleigh and Rician channels are difficult to model because delays cannot be accurately modeled in MAS, where “ticks” based delays are system related (for e.g. 1 tick = 1 frame/sec) and not procedure dependent.

Build time Easy to implement communication protocols and algorithms due to high level of abstraction provided by most of the MAS programming languages.

Build time is also higher because writing codes for communication protocols needs to be done from scratch.

Debugging Debugging is often easy. This is so because software like Netlogo provides access to the variables, command line debugging methods for printing the results and access to monitoring tools, all during the run-time.

Since MAS modeling represents evolutionary systems, the results that will be obtained later cannot be expected in advance.

Scalability of the model

Scalability is an added advantage in MAS. Scalability can be varied just by adding or removing nodes. Therefore adding new agents are easy than adding new capabilities to monolithic systems.

Again, up-scaling needs tweaking of computational power in large scale simulations (thousands of nodes).

Platform Most of the MAS simulators are platform independent because they are Java based. So they can run in virtually all the platforms and can make the best use of rich Java libraries.

If the system is time critical and requires lower level programming, Java is not the best candidate of selection unlike languages like C, C++.

63

7. REFERENCES

[1] Kumar A., Kaushik S.K., Sharma R., & Raj P., “Simulators for Wireless Networks: A Comparative Study,” in 2012 International Conference on Computing Sciences, 2012, pp. 338–342.

[2] Ivanov S., Herms A., & Lukas G., “Experimental Validation of the ns-2 Wireless Model using Simulation, Emulation, and Real Network,” Institute for Distributed Systems, Otto-von-Guericke University of Magdeburg, 2007. [Online]. Available: http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=5755511. [Accessed: 29-Oct-2014].

[3] “OMNeT++ Network Simulation Framework.” [Online]. Available: http://www.omnetpp.org/. [Accessed: 21-Sep-2014].

[4] “WhatIsGR - GNU Radio - gnuradio.org.” [Online]. Available: http://gnuradio.org/redmine/projects/gnuradio/wiki/WhatIsGR. [Accessed: 21-Sep-2014].

[5] “OpenBTS | Open Source Cellular Infrastructure.” [Online]. Available: http://openbts.org/. [Accessed: 21-Sep-2014].

[6] Wilensky U., “NetLogo Home Page,” 1999. [Online]. Available: https://ccl.northwestern.edu/netlogo/. [Accessed: 06-Sep-2014].

[7] Zavoral F., Jung J.J., & Badica C., “Intelligent Distributed Computing VII: Proceedings of the 7th International Symposium on Intelligent Distributed Computing - IDC 2013, Prague, Czech ... 2013,” Aug. 2013.

[8] Huhns M.N., Distributed Artificial Intelligence, Volume 1, 1st ed. Elsevier B.V, 2012, p. 390.

[9] Alonso E., “From Artificial Intelligence to Multi-Agent Systems: Some Historical and Computational Remarks.” [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.26.3210. [Accessed: 23-Sep-2014].

[10] Weiss G., Multiagent systems: a modern approach to distributed artificial intelligence, 3rd ed. MIT Press, 2000, p. 643.

[11] Patel M., Balakrishnan K., & Honavar V.G., Advances in the Evolutionary Synthesis of Intelligent Agents, 1st ed. MIT Press, 2001, p. 455.

64

[12] Rao A.S. & Georgeff M.P., “BDI Agents: From Theory to Practice,” Proc. FIRST Int. Conf. MULTI-AGENT Syst. (ICMAS-95}, pp. 312–319, 1995.

[13] Hyacinth N., “Software Agents: An Overview,” Intelligent Systems Research, Advanced Applications & Technology Department, BT Laboratories, Martlesham Heath, 1996. [Online]. Available: http://www.sce.carleton.ca/netmanage/docs/AgentsOverview/ao.html. [Accessed: 24-Sep-2014].

[14] Franklin S. & Graesser A., “Is it an Agent, or Just a Program?: A Taxonomy for Autonomous Agents,” in ECAI ’96 Proceedings of the Workshop on Intelligent Agents III, Agent Theories, Architectures, and Languages, 1996, pp. 21–35.

[15] Kubera Y., Mathieu P., & Picault S., “Everything can be agent!,” in AAMAS ’10 Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, 2010, pp. 1547–1548.

[16] Allen Kent and James G. William, Encyclopedia of Computer Science and Technology: Volume 42 - Supplement 27. CRC Press, 2000, pp. 30–32.

[17] Di Stefano B.N. & Lawniczak A.T., “Cognitive agents: Functionality & performance requirements and a proposed software architecture,” in 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH), 2009, pp. 509–514.

[18] “FIPA Agent Communication Language Specifications.” [Online]. Available: http://www.fipa.org/repository/aclspecs.html. [Accessed: 11-Sep-2014].

[19] Green M., “Speech Acts,” 03-Jul-2007. [Online]. Available: http://plato.stanford.edu/entries/speech-acts/. [Accessed: 24-Sep-2014].

[20] Holzer M. & Kutrib M., “Cellular automata and the quest for nontrivial artificial self-reproduction,” in CMC’10 Proceedings of the 11th international conference on Membrane computing, 2010, pp. 19–36.

[21] Bonabeau E., “Agent-based modeling: methods and techniques for simulating human systems.,” Proc. Natl. Acad. Sci. U. S. A., vol. 99 Suppl 3, no. 90003, pp. 7280–7, May 2002.

[22] Gilbert N. & Troitzsch K.G., Simulation for the Social Scientist. Open University Press, 2005.

[23] Macy M.W. & Willer R., “F ROM F ACTORS TO A CTORS : Computational Sociology and Agent-Based Modeling,” Annu. Rev. Sociol., vol. 28, no. 1, pp. 143–166, Aug. 2002.

65

[24] Ferguson T.S., “Game Theory,” Mathematics Department, UCLA, 2014. [Online]. Available: http://www.math.ucla.edu/~tom/Game_Theory/Contents.html. [Accessed: 25-Sep-2014].

[25] Bezek A. & Gams M., “Comparing a Traditional and a Multi-Agent Load-Balancing System,” COMPUTING AND INFORMATICS, vol. 25, no. 1. pp. 17–42.

[26] Nicholas R. Jennings M.W., “On agent-based software engineering,” Artif. Intell. Elsevier Sci. Publ. Ltd. Essex, UK, vol. 117, pp. 277 – 296, 2000.

[27] Zambonelli F., Jennings N.R., & Wooldridge M., “Developing multiagent systems,” ACM Trans. Softw. Eng. Methodol., vol. 12, no. 3, pp. 317–370, Jul. 2003.

[28] Ferber J. & Gutknecht O., “A meta-model for the analysis and design of organizations in multi-agent systems,” in Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160), pp. 128–135.

[29] North M.J., Collier N.T., Ozik J., Tatara E.R., Macal C.M., Bragen M., & Sydelko P., “Complex adaptive systems modeling with Repast Simphony,” Complex Adapt. Syst. Model., vol. 1, no. 1, p. 3, 2013.

[30] Foundation L., “The Logo Programming Language,” 2012. [Online]. Available: http://el.media.mit.edu/logo-foundation/logo/programming.html. [Accessed: 09-Sep-2014].

[31] Farrell J., Java Programming, 6th ed. Cengage Learning, 2011, pp. 1–976.

[32] Collier N. & North M., “Parallel agent-based simulation with Repast for High Performance Computing,” Simulation, vol. 89, no. 10, pp. 1215–1235, Nov. 2012.

[33] R Development Core Team, “R: A Language and Environment for Statistical Computing,” 2008. [Online]. Available: http://www.r-project.org.

[34] MATLAB, version 7.10.0 (R2010a). Natick, Massachusetts: The MathWorks Inc., 2010.

[35] Minar N., Y R.B., & Z C.L., “The Swarm Simulation System: A Toolkit for Building Multi-Agent Simulations,” 1996.

[36] Corp. I.B.M., “Eclipse Platform Technical Overview.” [Online]. Available: https://www.eclipse.org/articles/Whitepaper-Platform-3.1/eclipse-platform-whitepaper.html. [Accessed: 26-Sep-2014].

66

[37] Adelinde M. Uhrmacher D.W., Multi-Agent Systems: Simulation and Applications (Computational Analysis, Synthesis, and Design of Dynamic Systems). CRC Press, 2009.

[38] Brooks R., “A robust layered control system for a mobile robot,” IEEE J. Robot. Autom., vol. 2, no. 1, pp. 14–23, Mar. 1986.

[39] Russell S.J. & Norvig P., Artificial Intelligence: A Modern Approach, 2nd ed. Pearson Education, 2003.

[40] Brenner T., ed., Computational Techniques for Modelling Learning in Economics, vol. 11. Boston, MA: Springer US, 1999.

[41] Stallman R., “GNU General Public License,” 2007. [Online]. Available: https://gnu.org/licenses/gpl.html. [Accessed: 08-Oct-2014].

[42] Lingnau A. & Drobnik O., “Simulating Mobile Communication Databases with the Swarm system,” 4th Int’l Work. Pers. Wirel. Commun. Copenhagen, pp. 22–23, 1999.

[43] Mochocki B.C. & Madey G.R., “H-MAS : A Heterogeneous , Mobile , Ad-hoc Sensor-Network Simulation Environment,” Univ. Notre Dame Notre Dame, 46703, pp. 1–5.

[44] Odersky M., Spoon L., & Venners B., “Programming in Scala: A Comprehensive Step-by-Step Guide, 2nd Edition,” Jan. 2011.

[45] Lindholm T. & Yellin F., Java Virtual Machine Specification. Addison-Wesley Longman Publishing Co., Inc., 1999, p. 473.

[46] Bellifemine F., Caire G., & Greenwood D., Developing multi-agent systems with JADE, 1st ed. John Wiley & Sons, Inc, 2007, p. 300.

[47] Railsback S.F., Lytinen S.L., & Jackson S.K., “Agent-based Simulation Platforms: Review and Development Recommendations,” Simulation, vol. 82, no. 9, pp. 609–623, Sep. 2006.

[48] Helin H., “Supporting nomadic agent-based applications in the FIPA agent architecture,” University of Helsinki, 2002.

[49] FIPA, “Part 2, Agent Communication Language,” Foundation for Intelligent Physical Agents, 1997. [Online]. Available: http://fipa.org. [Accessed: 05-Oct-2014].

67

[50] Smith R.G., “The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver,” IEEE Trans. Comput., vol. C–29, no. 12, pp. 1104–1113, Dec. 1980.

[51] Masuoka R., Genesereth M.R., & Akira Sato, “KIF as a FIPA-CLL Content Language–An ontology for actions–(CFP7 001),” 1999. [Online]. Available: http://tsig.fujitsulabs.com/~rmasuoka/papers/kif-in-fipa-cll.pdf. [Accessed: 05-Oct-2014].

[52] Stephen Cranefield M.P., “Referencing Objects in FIPA SL: An Analysis and Proposal,” Proc. 2nd Int. Work. Challenges Open Agent Environ. AAMAS 2003, 2003.

[53] Schiemann B. & Schreiber U., “OWL DL as a FIPA ACL content language,” Artificial Intelligence Division, Department of Computer Science, 2006. [Online]. Available: http://wwwdh.informatik.uni-erlangen.de/download/jadeowlcodec/OWLasCLFIPAACL.pdf. [Accessed: 25-Oct-2014].

[54] McCarthy J., “History of LISP,” ACM SIGPLAN Not., vol. 13, no. 8, pp. 217–223, Aug. 1978.

[55] Finin T., Weber J., Wiederhold G., Genesereth M., Fritzson R., Mckay D., McGuire J., Pelavin R., Shapiro S., & Beck C., “DRAFT Specification of the KQML Agent-Communication Language,” University of Maryland, Baltimore County, 1993.

[56] Perdikeas M.K., Chatzipapadopoulos F.G., Venieris I.S., & Marino G., “Mobile agent standards and available platforms,” Comput. Networks, vol. 31, no. 19, pp. 1999–2016, Aug. 1999.

[57] Zhiyong F., Kai Y.U., Yang J.I., Ping Z., Li V.O.K., & Yongjing Z., “Multi-access Radio Resource Management using Multi-Agent System,” vol. 1, no. 60318004, pp. 63–68, 2006.

[58] Ashayeri M.R. & Taheri H., “Mobile WiMAX capacity estimation in Various Conditions,” in 2010 18th Iranian Conference on Electrical Engineering, 2010, pp. 483–488.

[59] Podder P.K., Islam F., Sarker D.K., Hasan M.G., & Kundu D., “AN ANALYTICAL STUDY FOR THE PERFORMANCE ANALYSIS OF PROPAGATION MODELS IN WiMAX.” [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.412.6897&rank=1. [Accessed: 05-Sep-2014].

68

[60] Mileo A., Schaub T., Merico D., & Bisiani R., “Knowledge-based multi-criteria optimization to support indoor positioning,” Ann. Math. Artif. Intell., vol. 62, no. 3–4, pp. 345–370, Aug. 2011.

[61] antenna-theory.com, “Friis Equation - (aka Friis Transmission Formula),” 2011. [Online]. Available: http://www.antenna-theory.com/basics/friis.php. [Accessed: 09-Sep-2014].

[62] Iraqi Y. & Boutaba R., “A Multi-agent System for Resource Management in Wireless Mobile Multimedia Networks,” pp. 218–229, Dec. 2000.

[63] Iraqi Y. & Boutaba R., “A Multi-agent System for Resource Management,” University of Waterloo, DECE, Waterloo, 2000. .

[64] Giri N. & Bodhe S., “ROLE OF MULTI AGENT SYSTEM FOR QOS GUARANTEE IN CELLULAR NETWORKS,” … J. Distrib. Parallel Syst., vol. 3, no. 5, pp. 59–72, 2012.

[65] Bigham J., “Agent-based resource management for 3G networks,” in Second International Conference on 3G Mobile Communication Technologies (3G 2001), 2001, vol. 2001, pp. 236–240.

[66] “Jade Site | Java Agent DEvelopment Framework.” [Online]. Available: http://jade.tilab.com/. [Accessed: 11-Sep-2014].

[67] Mengistu D., Tröger P., Lundberg L., & Davidsson P., “Scalability in Distributed Multi-Agent Based Simulations: The JADE Case,” in 2008 Second International Conference on Future Generation Communication and Networking Symposia, 2008, vol. 5, pp. 93–99.

[68] Morris A., “SAMSON : Strong Multi-Agent Simulation of Wireless Sensor Networks,” University of Edinburgh, 2008.

[69] Rao A.S., “AgentSpeak(L): BDI Agents speak out in a logical computable language,” Proceeding MAAMAW ’96 Proc. 7th Eur. Work. Model. Auton. agents a multi-agent world, pp. 42–55, 1996.

[70] Morris A., Giorgini P., & Abdel-Naby S., “Simulating BDI-Based Wireless Sensor Networks,” in 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, 2009, vol. 2, pp. 78–81.

[71] Calistru I.T., Cotofrei P., & Stoffel K., “A Clustering Topology for Wireless Sensor Networks New Semantics over Network Topology,” 2nd Int. Conf. Data Manag. Technol. Appl. (DATA 2013), pp. 153–160, 2007.

69

[72] Belal R.M. Bin, “Multi-hop Routing Scheme in Decentralized Wireless Networks ODD Protocol,” 2013. [Online]. Available: http://ccl.northwestern.edu/netlogo/models/community/WSN-Project-20-Final-6. [Accessed: 06-Sep-2014].

[73] Gupta N. & Das S.R., “Energy-Aware On-Demand Routing for Mobile Ad Hoc Networks,” Lecture Notes in Computer Science, 28-Dec-2002. [Online]. Available: http://link.springer.com/chapter/10.1007%2F3-540-36385-8_17. [Accessed: 21-Oct-2014].

[74] Brandl M., Posnicek T., Kellner K., Kos A., Mayerhofer C., & Fabian C., “A Simple Pheromone Based On-Demand Routing Protocol for Wireless Sensor Networks,” in 2009 Fourth International Conference on Systems and Networks Communications, 2009, pp. 254–260.

[75] Falconer D.D., Adachi F., & Gudmundson B., “Time division multiple access methods for wireless personal communications,” IEEE Commun. Mag., vol. 33, no. 1, pp. 50–57, 1995.

[76] Bhuyan P., “GSM: An Introduction.” [Online]. Available: http://www.angelfire.com/indie/pradip_bhuyan/gsm/GSM.html. [Accessed: 11-Sep-2014].

[77] Fullmer C.L. & Garcia-Luna-Aceves J.J., “Solutions to hidden terminal problems in wireless networks,” in Proceedings of the ACM SIGCOMM ’97 conference on Applications, technologies, architectures, and protocols for computer communication - SIGCOMM '97, 1997, vol. 27, no. 4, pp. 39–49.

[78] Amouris K., “Space-time division multiple access (STDMA) and coordinated, power-aware MACA for mobile ad hoc networks,” in GLOBECOM’01. IEEE Global Telecommunications Conference (Cat. No.01CH37270), 2001, vol. 5, pp. 2890–2895.

[79] Gonzalez A., Marshall I., & Sacks L., “A Self-Synchronised Scheme for Automated Communication in Wireless Sensor Networks,” in Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004., 2004, pp. 97–102.

[80] Puri Novelti Anggraeni and Satya Ardhy Wardana, “Cooperation in Wireless Grids Protocol for Cooperative Network with Game Theory Model,” 2007.

[81] “IEEE-SA -IEEE Get 802 Program - 802.11: Wireless LANs.” [Online]. Available: http://standards.ieee.org/about/get/802/802.11.html. [Accessed: 14-Sep-2014].

70

[82] Aissa I., Frikha M., & Tabbane S., “Dynamic Spectrum Hole Management in Cognitive Radio,” in 2011 4th IFIP International Conference on New Technologies, Mobility and Security, 2011, pp. 1–4.

[83] Fette B., Cognitive radio technology. Elsevier Inc., 2009, pp. 73–118.

[84] Mitola J., “Cognitive Radio---An Integrated Agent Architecture for Software Defined Radio,” Royal Institute of Technology (KTH), 2000.

[85] Mitola J., “Cognitive radio for flexible mobile multimedia communications,” in 1999 IEEE International Workshop on Mobile Multimedia Communications (MoMuC’99) (Cat. No.99EX384), 1999, pp. 3–10.

[86] Cattoni A.F., “Module 2 - Overview on CR Topics: Dynamic Spectrum Access,” 2012. [Online]. Available: https://www.youtube.com/watch?v=80qWzp_PSc8&list=UULLpXGUQ_SlcSWAnMiZ2Clw. [Accessed: 16-Sep-2014].

[87] Trigui E., Esseghir M., & Boulahia L.M., “ON USING MULTI AGENT SYSTEMS IN COGNITIVE RADIO NETWORKS,” Int. J. Wirel. Mob. Networks, vol. 4, no. 6, 2012.

[88] Trigui E., Esseghir M., & Boulahia L.M., “Cognitive radio spectrum assignment and handoff decision,” in 2013 IEEE 24th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), 2013, pp. 2881–2886.

[89] Horine B., Boloni L., & Turgut D., “Distributed decision making in cognitive radio networks through argumentation,” in 2013 IEEE Global Communications Conference (GLOBECOM), 2013, pp. 1209–1214.

[90] Miliotis V., Koutsopoulos I., Alonso L., & Verikoukis C., “Low-overhead Heuristic Algorithms for Spectrum Sensing in Cognitive Radio Networks.” pp. 1–6, 2014.

[91] Grace D., “Reciprocal learning for cognitive medium access,” in 2013 IEEE Wireless Communications and Networking Conference (WCNC), 2013, pp. 89–94.

[92] Li H., “Multi-Agent Q-Learning for Competitive Spectrum Access in Cognitive Radio Systems,” in 2010 Fifth IEEE Workshop on Networking Technologies for Software Defined Radio Networks (SDR), 2010, pp. 1–6.

71

[93] Mitchell P.D., Jiang T., & Grace D., “Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing,” IET Commun., vol. 5, no. 10, pp. 1309–1317, Jul. 2011.

[94] Mir U., Merghem-Boulahia L., & Gaïti D., “A Cooperative Multiagent Based Spectrum Sharing,” in 2010 Sixth Advanced International Conference on Telecommunications, 2010, pp. 124–130.

[95] Wong V.W.S. & Leung V.C.M., “A Coalition Formation Game for Energy-Efficient Cooperative Spectrum Sensing in Cognitive Radio Networks with Multiple Channels,” in 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011, 2011, pp. 1–6.

[96] Santos P.M.R. dos, Kalil M.A., Artemenko O., Lavrenko A., & Mitschele-Thiel A., “Self-organized common control channel design for Cognitive Radio Ad Hoc Networks,” in 2013 IEEE 24th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), 2013, pp. 2419–2423.

[97] Fitzek F. & Katz M., Cognitive wireless networks: concepts, methodologies and visions inspiring the age of enlightenment of wireless communications. Springer Publishing Company, 2007, p. 750.

[98] Dzikowski J. & Hood C., “An agent-based simulation framework for cognitive radio studies,” in Proceedings of the Second International ICST Conference on Simulation Tools and Techniques, 2009, p. 18.

[99] Dorleus J., Holweck R., Ren Z., Li H., Cui H.-L., & Medina J., “Modeling and Simulation of Fading and Pathloss in OPNET for Range Communications,” in 2007 IEEE Radio and Wireless Symposium, 2007, pp. 407–410.

[100] Polley J., Blazakis D., McGee J., Baras J.S., & Karir M., “ATEMU: a fine-grained sensor network simulator,” in 2004 First Annual IEEE Communications Society Conference on Sensor and Ad Hoc Communications and Networks, 2004. IEEE SECON 2004., 2004, pp. 145–152.

[101] “WSNet simulator for large scale wireless sensor networks.” [Online]. Available: http://wsnet.gforge.inria.fr/. [Accessed: 20-Sep-2014].

[102] Lim H.B., Wang B., Fu C., Phull A., & Ma D., “WISDOM: Simulation Framework for Middleware Services in Wireless Sensor Networks,” in 2008 5th IEEE Consumer Communications and Networking Conference, 2008, pp. 1269–1270.

72

APPENDIX A- RADIO MODEL

extensions [matrix] extensions [array] globals [ FSPL Dist_m SNR BER EbNo Lp] breed [BSs BS] BSs-own [power-in-dB] breed [nodes node]

nodes-own [ xcord ycord dist ] breed [ timers a_timer ] timers-own [ owner timeout action ] ;; Setup the radio environment to create clear-all create-BSs num-BS [ set shape "bst" set color Red set

size 20 set power-in-dB -30] create-nodes num-nodes [ set shape "person" set color 52 set label who set label-color red ] reset-ticks end ;; Perform Calculation to calculate ;let dist Distance-nodes ask BSs [set ycor 0 set xcor 0]

73

ask nodes [ ; head in a random direction perform-mobility ;; subroutine for mobility ] tick ask node Node_of_interest [ ask patch-here [ set pcolor yellow ] print word("The distance from the node ")

Node_of_interest print word(" to the Base Station is " ) distancexy 0 0

;; Initialize Parameters and calculate Free Space

Path Loss set Dist_m distancexy 0 0 if Dist_m > Comm_range [ set label " Out of Range!" wait 2 type "the node has gone out beyond the range

of communication. " set label who stop ] let c 3 * 10 ^ 8 let freq 0 set freq Frequency_MHz let lambda 0 set lambda c / (freq * 1000000)

74

set FSPL 20 * log( Dist_m) 10 + 20 * log(freq)10 +

32.45 print word("The Free Space PathLoss is ")FSPL ;;link budget assignments Gains in dBi Losses in

dB let Pt 0 let B Bandwidth_MHz_receiver * 10 ^ 6 let k 1.380650 * 10 ^ -23 let NF 7 let Pr 0 set Pt Tx_Power_dBm let Lcr 0.56 ;;Effective Noise Temp all in Kelvins let Te 0 let T0 290 let Tant 300 set Te 290 print word("Power Transmitted is ")Pt print"dBm" set Pr (Pt - FSPL) Print word("The power received is ")Pr print"dB"

;; 30 dB fade margin Let Np (10 * log(k * (Te) * B)10) + 30 + NF Print

word("The noise power obtained is ")Np print"dBm" set SNR Pr - Np print word("The received SNR is ")SNR ;;EbNo Calculation let Eb 0 set Eb Pr + ( 10 * log(B)10) print Eb let No 0 set No ( Np + 10 * log(1000000)10) print

No set EbNo (Eb - No) print word("The Eb/No for the

bit rate 512 kbps system is ")EbNo print "dB" ;;erfc calculation let z sqrt(EbNo)

75

let t 1.0 / (1.0 + 0.47047 * abs(z)) let ans 0 let erfc 0 set erfc exp( - (z * z)) * ((0.3480242 * t) + (t ^

2 * ( -0.0958798)) + (t ^ 3 * ( 0.7478556))) ;; Abramowitz and Stegun approximation

print word("the value of erfc is ")erfc ;;BER calculation for BPSK set BER 0.5 * erfc print word("The BER for BPSK transmitted signal is

")BER do-exhibition1 ] clear-patches reset-ticks end to perform-mobility rt random-float 360 set xcord xcord + (transition * dx) set ycord ycord + (transition * dy) ; hide turtles who have moved out of the world

ifelse patch-at (xcord - xcor) (ycord - ycor) = nobody

[ ht ] [ st set xcor xcord set ycor ycord ] set dist sqrt (xcord * xcord + ycord * ycord) end to terminate

76

ask node Node_of_interest [set label "" ] end to do-exhibition1 if not plot? [ stop ] set-current-plot "FSPL" set-plot-pen-interval 1 set-plot-x-range 0 30 set-plot-y-range 50 150 plotxy Dist_m FSPL end to-report Received_SNR report SNR end to-report BER_BPSK report BER end to-report Eb/No report EbNo end

;; Initialize Parameters and calculate Okumura-Hata Loss Model (Urban)

to calc ask nodes [ perform-mobility] ask node Node_of_interest [ print word("The distance from the node ") Node_of_interest print word(" to the Base Station is " ) distancexy 0 0 let d distancexy 0 0 let f Frequency_MHz

77

let Hb Height_BS_meters let Hm Height_Mobile_meters let A 0 let B 0 let a_Hm 0 set a_Hm 3.2 * (log(11.75 * Hm)10) ^ 2 - 4.97 set A 69.55 + 26.16 * log (f)10 - 13.82 * log(Hb)10 - a_Hm set B 44.9 - 6.55 * log(Hb)10 set Lp 0 set Lp A + B * log(d)10 Print word("The Okumura-Hata Path Loss for the distance " )d Print word("is ") Lp do-exhibition2 ] tick end to do-exhibition2 set-plot-pen-interval 1 set-current-plot "Okumura-Hata" plot (Lp) end

;ask turtles [forward 12] ;ask turtles [set heading 32] ;ask turtles [ setxy random xcor ycor] ;ask turtles [ setxy random xcor ycor pen-down left 45

forward 2] ;ask turtles with[ ycor != 0][ set shape "person" set

ycor 0 set xcor 10 ; ] ;ask turtles with [hidden?][ ;set color orange] ;;//Noise Generation AWGN// to noise

78

reset-ticks let sigma 0.7 ;;standard deviation of noise let i 0 ;; signal of concern ;; noisy signal let t ticks let x array:from-list n-values 2000 [0] let n array:from-list n-values 2000 [0] let y array:from-list n-values 2000 [0] while[i < 2000] [ array:set n i sqrt sigma * random-normal 0 1 array:set x i sin (2 * pi * Frequency_MHz * i) set i i + 1 ] let xl array:to-list x let nl array:to-list n let yl array:to-list y set yl (map [ ?1 + ?2 ] xl nl) print word("Value of x is ")xl print word("Value of n is ")nl print word("Value of y is ")yl tick end

79

APPENDIX B- GSM TDMA to TDMA clear-all

create-BSs 1

ask BSs [ set ycor 0 set xcor 0 set shape "bst" set color Red set size 20 set power-in-dB -30]

let i 0; set framedur 0; set Slot 0.577;

while[i < num-nodes AND framedur <= 4.61538] [ crt 1 [setxy 1 2 forward 10 setxy random xcor random ycor set shape "person" set color 52] reset-timer; while[timer <= Slot] [ print word(" CALCULATION FOR NODE ")(i + 1) ask turtle ( i + 1 ) [create-link-with turtle 0] ask links [set color blue] let a [xcor] of turtle i + 1 let b [ycor] of turtle i + 1 let hyp sqrt(a ^ 2 + b ^ 2) set Dist hyp * 1000; calculate //Subroutine call ] ask links with [color = blue] [die]; set i i + 1; set framedur framedur + timer; print word(" FRAME DURATION ")framedur; ]

end to-report FrameDuration report framedur end

80

APPENDIX C- PRIORITY BASED SLOT ASSIGNMENT extensions [matrix] extensions [array] globals [ FSPL Dist_m SNR BER EbNo Lp pr-list datarates num_voice num_real num_data] breed [BSs BS] BSs-own [power-in-dB] breed [nodes node] nodes-own [ xcord ycord dist priority ] breed [ timers a_timer ] timers-own [ owner timeout action ] ;; Setup the radio environment to create clear-all create-BSs num-BS [ set shape "bst" set color white set size 20 set power-in-dB -30] create-nodes num-nodes [ set shape "mob" set size 2 set color 52 set label who set label-color red]

81

reset-ticks end to set-priority

;;set the priority on a random basis set datarates ["96kbps" "512kbps" "64kbps"] let i 1; set pr-list []; while [i < num-nodes] [ ask node (i) [set priority random 3 fd 0.2] set i i + 1 ] set pr-list ([priority] of nodes) end to assign set datarates ["96kbps" "512kbps" "64kbps"]

foreach (pr-list) [ask nodes with [priority = 0] [ifelse ? = 0 [voice] [fd 1]] ] foreach (pr-list) [ask nodes with [priority = 1] [ifelse ? = 1 [realtime-video] [fd 1]] ] foreach (pr-list) [ask nodes with [priority = 2] [ifelse ? = 2 [store-forward-data] [fd 1]] ] end to voice set num_voice count nodes with [priority = 0] ask n-of (num_voice - 1) nodes with [priority = 0]

[create-link-with turtle 0 fd 0.1] ask links[ set color green set label item 0

datarates] ask links with [color = green] [die]

82

end to realtime-video set num_real count nodes with [priority = 1] ask n-of (num_real - 1) nodes with [priority = 1]

[create-link-with turtle 0 fd 0.1] ask links[ set color orange set label item 1

datarates] ask links with [color = orange] [die] end to store-forward-data set num_data count nodes with [priority = 2] ask n-of (num_data - 1) nodes with [priority = 2]

[create-link-with turtle 0 fd 0.2] ask links[ set color red set label item 2 datarates] ask links with [color = red] [die] end ;; Perform Calculation to calculate ;let dist Distance-nodes ask BSs [set ycor 0 set xcor 0] ask nodes [ ; head in a random direction perform-mobility ;; subroutine for mobility set-priority assign voice realtime-video store-forward-data ] tick end