Ad Hoc Routing Protocols: Emulation vs SimulationA. Giovanardi, G. Mazzini

DI, University of Ferrara, ItalyE-mail: {agiovanardi, gmazzini}@ing.unife.it

Abstract- This paper presents an emulation platform obtainedas extension of the Simple Ad hoc siMulator (SAM) [1] (wheremany routing protocols for ad hoc networks are implemented)and discusses the results obtained when the OLSR routing proto-col is considered. Emulation can give a controlled, reproducibleenvironment for running live code in a small, lab-environmentnetwork. Furthermore, the benefits of network emulation includethe ability to expose experimental algorithms and protocols tolive traffic loads, and to introduce real packet processing times.Finally, in many cases, a large number of hosts can be consideredwithout impact on the final cost. With respect to a wireless livetest, the emulator used in this paper works on hosts connectedeach other via wired links and the wireless channel is simulated.A direct comparison between simulation and emulation resultsis presented to underline the benefits of emulation.

I. INTRODUCTIONAd hoc networks have become an increasingly popular

technology in the past few years: nodes communicate withother nodes in their range and act as forwarders of datafrom nodes communicating with out-of-range nodes. In theliterature, many simulators (OPNET [2], NS2 [3], GloMoSim[4], SAM [1]) and emulators ([5] [6] [7] [8]) have beenproposed and developed to validate the proposal of new routingprotocols for ad hoc networks.

Emulation environments for ad hoc networks involves acertain number of computers (hosts) equipped with wiredand/or wireless cards. These hosts exchange packets betweenthem and some system behaviors are simulated, such astopology, mobility and in many cases also propagation channel(even if a wireless card is considered). In fact, the use ofmobility and propagation channel synthetic models allows totest many system situations and to have trials and resultseasy to reproduce. To correctly manage on different computersthe simulated system characteristics often a central serveris present. In some cases [6] all traffic is directed via thatserver, which forwards that traffic to the destinations (CentralControl Emulators). In other emulation architectures [5] [7] thedata packets do not passes through the server, which simplymanage the whole emulation. Finally, in other proposals eachstation acting as mobile node is responsible for directing andforwarding traffic, i.e., no one central server is necessary [8](Distributed Emulators).

Emulation can be classified also regarding its easy softwareportability or not. Many of the proposals need the modificationof the kernel code of the operating system or to load suitablekernel modules, with a consequent not easy portability.

In this paper we test the Optimized Link State Routing(OLSR) [9] routing protocol with the emulator platform pro-

This work is developed under Regional Insebala Project.

0-7803-9206-X/05/$20.00 ©2005 IEEE140

1/0 events

5vnNew(id.MAC,N5W_PKTIN_Q.cIock+gr:ion_i,,e)

Even.Ne(id,ROUTE,PKT-FOR-ROUTE,cIok) Internal eve

Ev-nN-6(id,ROUTE.PKT_READY FOR ROULTEC,o})

nts

- emulation routing signaling traffic-b. data packet forwarding

basic traffic (tcp,udp,arp,multicast,broadcast,..)

Fig. 1. Emulation test bed.

posed in [10] [11]. The emulator is falling in the DistributedEmulators category and is easy to implement (the architectureis fully developed in the user space, i.e., it does not need anychange to the kernel code). The structure is fully distributed,and the nodes exchange signaling routing packets (according tothe selected routing protocol) to update topology knowledge.On the basis of the run-time updated routing tables, the datapackets are transmitted from the source to the destination withhops on intermediated nodes. The nodes are connected viawired links and the wireless channel is simulated.

Emulator interfaces with SAM [1], where many kindsof unicast and multicast ad hoc routing protocols (Dijkstra,Link State, Distance Vector, DSR, AODV, OLSR, AMRIS,ODMRP, AMRoute) are present. This interface allows toperform direct comparison between simulation and emulation,in order to underline main advantages of emulation in theperformance investigation of routing protocols for ad hocnetworks. While in [10] [11] we have presented the emulatorimplementation details and we have reported results involvingDijkstra and Link State routing protocols, in the present paperwe focus the attention on the comparison of the emulation re-sults with the simulation ones in order to underline differences,by considering OLSR protocol.

II. EMULATOR DESCRIPTIONThe emulator has been developed and tested on a cluster of

Linux hosts connected by means of a Fast Ethernet network,but the framework is general, so it can be implemented on

Fig. 2. Graphic interface screenshot. "Connected" links are depicted withred lines.

other data link layer types, such as the IEEE 802.11 of awireless interface. To use the routing protocols still present inSAM, a suitable interface with this simulator has been created,so that many of the software blocks have been integrated inthe emulator. The emulation architecture considers a networkcomposed by N hosts and each of them has routing functions,i.e., can relay packets to other nodes.

In Figure 1 the emulation scenario is depicted. The hostsare involved in many activities (not only emulation) and so abasic real traffic is always present (relative to services andapplications such as NFS, DNS, DHCP, multicast and soon). The emulation traffic is composed by signaling broadcastpackets (useful to know and update the topology) and bydata packets. In Figure 1 an example of packet forwardingperformed by the host number 3 from source 1 to destinationN is showed. The Figure shows also the presence of internalevents (relative to the natural evolution of the SAM simulation)and of 1/0 events (relative to the incoming of data andsignaling packets at the network interface). A suitable eventscheduler have been inserted to coordinate the presence ofthese two kinds of events.SAM is a discrete event simulator, composed by the follow-

ing modules: Traffic, dealing with the traffic generation; Chan-nel, simulating the radio propagation; Mob, dealing with theterminal mobility; Radio, simulating the hardware transceiver;Mac realizing the Data Link Layer; Route, implementingthe Network layer; Statistics, collecting statistics results. Ourinterface allows to interact with the Traffic, Channel, Mob andRoute modules. Furthermore, for the sake of simplicity wehave not inserted the TCP at the Transport Layer, by onlyconsidering UDP.

Emulator includes a software structure managing the net-work interface: on each host, the packets are dumped by usingthe libpcap library [12] and are inserted on the network byusing the RAW sockets [13]. The broadcast routing signalingpackets has been redirected to a MAC address different fromthe broadcast one, to avoid to flood the network with broadcastpackets not interesting for the hosts not involved in the emu-

lation. To discriminate into the network the packets involvedin the emulation from those relative to other applications orprotocols, we have identified and selected an ad hoc typefield in the Ethernet frame (0x0020). A TIL is includedinto the frame to drop a packet circulating into the networkindefinitely (when TIL reaches 0 the packet is eliminated);furthermore, TTL is useful to evaluate the total number of hopsexperimented by a given packet. To insert the data structuresinto the frames in a machine-independent fashion, the eXternalData Representation standard (XDR) [14] has been adopted.To do this the XDR library routines available under Linuxhave been used.To generate packets into the network the module Traffic

present in SAM has been adopted. This module presents var-ious kind of traffics: Poisson, CBR, Isochronous, FTP, Video;and, by default, the number of active nodes (N.) is set equalto N, i.e., all hosts are generating traffic. At the frame born,the source node n, collects the generation time, to identify thetimestamp to include in that frame, in order to collect deliverytime. Then, ns, knowing the destination host nd, searches thenexthop in the path, by applying the rules corresponding to theselected routing protocol. If the nexthop exists, n, creates theframe to send, by setting in the Ethernet header: SA= SourceMAC, DA= Nexthop MAC, Type= 0x0020; and including ns,nd, TTL, timestamp, nexthop and transmit power. After theframe creation, n, sends the frame to the selected interfaceby using the RAW socket. The frame forwarding capability(router functionality) is started on each host and is maintainedactive until the end of the emulation. In particular, when a hostcaptures a frame with type field 0x0020, it can discriminateif this frame is addressed to another node or not. In thefirst case it does not perform any action; in the last caseit processes the frame to establish if it must be forwardedto the next hop or it has reached the final target. The firstoperation of the frame processing is to identify the source anddestination nodes, the TTL and the timestamp. Then, also thelast node in the path which has relayed the frame is identified(it is often an intermediate node, i.e., not the source node).This permits to evaluate the channel impairment in the lastlink, and then, according to a received power threshold, todetermine if the frame is correctly received or not. If theframe is correct and has not reached the final target, the localnode starts the procedures to forward it to the next hop. If theframe is correct and has reached its last destination, the localnode processes it to collect performance indexes. Otherwisethe frame is dropped.To implement and support mobility, the module Mob present

in SAM has been adopted. This module, at scheduled timeinstants (according to a selected movement model), recomputethe position of all mobile hosts (which can be a subset of N).In the emulation each host has not direct information on thenew location assumed by the other nodes in the network, infact each host executes only its own steps, according to theinternal and external events. To solve this problem at eachposition update each node sends to all other nodes informationon the new position assumed by itself. Since the connectionare very fast the latency in the position upgrade is very low.Many trials confirm the good synchronization in the node

141

[ _________________ [I Ptx =-28 J Pt2 =-27 ] Ptx =-26 IPtx =-251 Ptx =-24 [ Ptx =-23 J Ptx =-22 Ptx =-21 ]PATH LOSS (EMUL) 160 169 173 267 242 [ 288 266 402PATH LOSS (SIM) 0.42 0.42 0.42 0.64 0.5 [ 0.9 0.9 6

PATH LOSS MOB (EMUL) 266 246 331 252 262 254 425 305PATH LOSS MOB (SIM) 123 J 187 ] 195 126 159 [ 129 [ 113 256SHADOW (EMUL) 391 400 453 455 474 424 379 375SHADOW (SIM) 4.34 4.34 7.81 9.06 6.85 9.91 6.30 6.26

SHADOW MOB (EMUL) 481 271 238 250 280 272 400 280SHADOW MOB (SIM) 109 149 202 230 233 332 355 279

TABLE IDELIVERY TIME (MS), EMULATION VS SIMULATION, BY VARYING Ptx (DBM), WITH DIFFERENT CHANNEL AND MOBILITY BEHAVIORS.

position information. A similar approach has been adoptedby [7], where hosts exchange mobility information through amulticast traffic.The wireless channel is simulated, even if a wireless card

could be used, as physical network interface. The choice ofsynthetic models for the wireless channel allows to easy repro-duce and test many system situations. In some cases in fact,could be hard reproduce topology behavior with nodes visibleby a selected subset of other nodes and to reproduce results inenvironment affected by conditions varying in unpredictableways. Thus, the module Channel present in SAM has beenadopted, by considering in particular path loss and log-normalshadowing as propagation models. When a node receives apacket it can compute the link attenuation (the transmissionpower is reported in the packet and the knowledge of theposition of the sending node is sufficient to compute thereceived power, according to the propagation models presentin Channel), and then establish if the packet is correct or not.To easy visualize the mobile host movements and the link

"connected" and "unconnected" which are established duringan emulation trial, a graphic interface as been created. Thisinterface is based on the Scripted Display Tool (SDT) tool[15] used also by [7]. In Figure 2 a graphic example withN = 10 nodes is reported (in red the connected links). Theconnection active between nodes are computed on the basisof the synthetic propagation models present in Channel.A suitable post-processing structure has been developed

to collect: success probability (correctly delivered packetfraction), delivery time (end-to-end time), number of hopsin the path, number of isolated nodes, energy spent perbyte, total energy spent for each single host and the relativeaverage values on the network.

III. NUMERICAL RESULTSWe have tested OLSR [9], a classic ad hoc routing protocol

falling in the proactive category.The wireless nodes are located in a square room with size

10xlOm. The propagation channel behavior is characterizedby path loss, with exponent d = 2.5 and reference distancedref = 0.2m. In some tests also log-normal shadowing withdeviation o- = 5dB has been considered. The minimumreceived power over which the packet transmission is as-sumed correct is Pr = -76dBm. The traffic is Poissonianwith average arrival rate A = 10packetls. The results in allFigures are obtained by considering 1000 packets generated

per station. In the performance evaluation we have variedthe number of nodes present in the network (in the setN = 4 6. 8, 10, 12,15, 20) and the transmit power Pt, in therange [-28, -21]dBm. All Figures show performance indexesrepresenting values averaged on the whole network.The first results (Table II and Figures 3, 4) are obtained

with N = 10 and by considering many different channel andmobility behavior:

1) all nodes fixed (no mobility), with only path loss (PATHLOSS);

2) 6 fixed nodes and 4 mobile nodes, with only path loss(PATH LOSS, MOB);

3) all nodes fixed (no mobility), with path loss and shad-owing (SHADOW);

4) 6 fixed nodes and 4 mobile nodes, with path loss andshadowing (SHADOW, MOB);

The mobile nodes move with a pseudo-linear trend withaverage speed, Vmob = 1.38 m/s (5 km/h, pedestrian); mobilityrefresh time, tmob = 0.3 s; velocity deviation, 0mob = 0.3 m/s;maximum mobility angle, 0 = 7r rad.

Table II shows the delivery time (end-to-end delay) inall preceding scenarios by varying Pt,, both for emulationand simulation. Note that the packet delivery time shouldinclude: packet transmission time, propagation delay, queuetime, packet processing time (computed on each link of themultihop path). In the SAM simulation the delivery timeis computed as difference between the simulation clock atthe packet delivery and the simulation clock at the packetgeneration, having included, in the event scheduling, onlypacket transmission times and propagation delays on each link,without consider queue and processing times.On the other hand, in the SAM emulation the delivery

time is computed as difference between the system time(collected with gettimeofday()) when the packet transmissionstart (through the Ethernet network interface) and the systemtime when the packet is dumped on the network. Thus itincludes also queue and packet processing times at each node,i.e., is a realistic time. As expected (as a consequence of thepresence of queue and processing times), by examining TableII we can note that in the emulation the delivery times arealways greater than in the simulation. Furthermore, simulationshows quite different times passing from fixed to mobilescenarios, even if the average number of hops in the pathis quite the same in the two cases (as reported in Figure4). This effect is not present in the emulation, showing the

142

0.5 P

um 0.3

0.2 [

0.1

PATH LOSS, emul ------PATH LOSS, sim *

PATH LOSS, MOB, emul -------PATH LOSS, MOB, sim K>

SHADOW, emul .........SHADOW, sim E

O. ,,,,.2 ..°,,,,,.,

................~~~~~~~~~~,.I .~~~~~~~~~~~~~~~~~~~~~

-.:-

.-------

................ --,,,--'°..........'

+

8 -27 -26 -25 -24 -23 -22 -2Ptx

Fig. 3. Psuc, as a function of Pt, by varying channel behavior andconsidering fixed and mobile nodes (SIM and EMUL).

0.35

0.3

0.2

0.15

0.1

0.05'

4 6 8 10 12 14

Fig. 5. P as a function of N, by varying channel behavior, by consideringPt = -22, -25dBm and 4 mobile nodes (SIM and EMUL).

.. ''',

'S --

I U'--

-27 -26 -25 -24 -23 -22

0.7

0.6

0.4

0.3 [

0.2 [

0.1

o2t-2f-21

Fig. 4. Hop number as a function of Pt2, by varying channel behavior andconsidering fixed and mobile nodes (SIM and EMUL).

same delivery time trend in all cases. Only in the scenario3) (SHADOW) the emulation time values are slightly greaterthan in scenarios 1) 2) and 4) as a consequence of an highernumber of hops in the path (as reported in Figure 4). Thus,the results confirm how the simulation cannot fully capturethe system performance, since its implementation may differfrom real one, as a consequence of the difficult to representthe real behavior by means of a synthetic environment.

Figure 3 shows the success probability i.e., the correctly de-livered packet fraction, as a function of PtX, having considered1), 2) and 3) scenario. The emulation results are plotted withlines, the simulation results with points. For these parametersettings a good match can be verified. Any way in the scenario4) (not plotted, for sake of simplicity) a performance gap

is present: in the simulation the success probability is quitedouble with respect to emulation (for all Pt. values). This isa consequence of the time difference in the packet reception(higher delivery delay in the emulation). In fact, in case ofmobility, if a packet needs more time to be delivered, it ispossible that the destination node has become unreachablewith higher probability with respect to lower delivery times. In

-27 -26 -25 -24 -23 -22Ptx

-21

Fig. 6. Ps1,,,, as a function of Pt,, by varying N = 10, 15, 20, only pathloss and fixed nodes.

case of pedestrian mobility the effect in presence of only pathloss is not so relevant, but if also shadowing is present a littlemismatch in destination location can cause high differencein the channel behavior with higher packet loss. In case offixed nodes the emulation and simulation results are perfectlymatching. Furthermore, the shadowing presence makes the net-work more dense and limits the portion of isolated networks,by making "reachable" nodes which are far each others, witha consequent higher success probability.

In figure 4 the average number of hops needed to reachthe destination is depicted for all 1), 2), 3) and 4) scenarios.As reported above, in case of shadowing with fixed node, thenumber of hops is higher than in the other scenarios, since thenetwork becomes more dense (with farer hosts reachable withhigh hop number) and the mobility mismatch does not affectthe system. In this case, a good match between emulation andsimulation is present.

In Figure 5 PSUCC as a function of N, by consideringPtx = -22,-25dBm, is reported. The scenarios consideredhave two channel behaviors: path loss or shadowing with 4mobile nodes. Note that emulation and emulation give quite

143

PATH LOSS, MOB, Ptx=-22dBm, emul -PATH LOSS, MOB, Ptx=-22dBm, sim *PATH LOSS, MOB, Ptx=-25dBm, emulPATH LOSS, MOB, Ptx=-25dBm, sim <>SHADOW, MOB PbS=-25dBm, emulSHADOW, MOB, Ptx=-25dBm, sim *

4,.. _*~~~~,''

PATH LOSS, emul -----PATH LOSS, sim *

PATH LOSS, MOB emulPATH LOSS, MOB, sim O

SHADOW, emulSHADOW, sim 2

SHADOW, MOB, emulSHADOW, MOB, sim U

....

I * ^~~

35

3

2.5

2

1.5

14

0 _.-28

PATH LOSS, N=10, emul .........PATH LOSS, N=10, sim *PATH LOSS, N=15, emul.PATH LOSS, N=15, simPATH LOSS, N=20, emul -------PATH LOSS, N=20, sim *K "I <

I, O

,""' .....

.......

...........- ...

z

I.8U)a. 4

.........t

'8

0.6

0.4

-28 -27 -26 -25 -24 -23 -22 -21Ptx

Fig. 7. Ni/N as a function of Pt1, by varying N = 10, 15, 20, only pathloss and fixed nodes.

some results, even if by increasing N the performance gapincreases. As further explanation of the difference betweenemulation and simulation, in particular in case of mobility, itis possible to note that the higher packet delivery times affectalso signaling packets. Thus, a lower success probability withrespect to simulation could be referred to a minor reactivityto a node position change, since the update of the routingtables needs to exchange and receive signaling packets (suchas HELLO or TC messages [9]) which income more late inemulation as a consequence of the presence of queue andprocessing times not considered in simulation. Finally, thesuccess probability increases both by increasing Pt, and N,as expected, since the network behaves more dense with morenodes in visibility.

In Figure 6 the average success probability, P511,,,, is re-

ported by considering the scenario with path loss and fixednodes, with N = 10,15, 20. For these settings the matchbetween emulation and simulation is quite good, even if byincreasing the number of nodes some performance gaps arepresent. As expected, by increasing the transmit power, thesuccess probability increases, since the probability to haveisolated nodes decreases. By increasing the number of nodesin the network, the effect is similar to a transmit powergrowth, since the network becomes more dense and then nodesare closer each one (and then more reachable) with higherprobability.

Let us note that emulation can also give results on perfor-mance indexes which are not available with SAM simulation.An example is the number of isolated nodes, i.e., the numberof nodes far from each other host and then not able to receiveor send packets form/to any other node. The knowledge ofthe number of isolated nodes is useful to derive if the valuesaveraged on the whole network are really representative of thenetwork of it is the case to consider in the final performanceonly nodes which are able to exchange packets. As an examplein Figure 7 the ratio between the number of isolated nodes,Ni, and the total number of hosts present in the network, N, isreported, by varying Pt, and by considering the scenario withpath loss and fixed nodes, with N = 10, 15, 20. As expected,

Ni/N decreases by increasing Pt, and by increasing N, i.e.,by increasing the density of nodes in the network area.

Let us underline that emulation gives final results which arevarying run by run. In fact, the presence of live traffic on thenetwork affects the final performance. In particular, the finaldelivery time can be particularly affected by different networktraffic load. The fluctuations on the performance parameters(such as the success probability and delivery time) are notso relevant, but they prove the effect of real environmentconditions that can not be easily captured and simulated ina synthetic environment such as simulation.

Finally, the emulation architecture can be transparently usedwith Linux hosts equipped with wireless network cards, asalso proposed in [7], where the emulator testbed can beimplemented both with wired and wireless cards. This allowsto consider also the behavior of a MAC protocol specificallydesigned for ad hoc networks (such as IEEE 802.1 1).

IV. CONCLUSIONSThe paper shows some results obtained by using the SAM

tool, used as emulator and simulator. The aim is to explainhow the emulation architecture can better describe the protocolfunctioning with respect to pure simulation. In fact, the pres-ence of real environment components, such as real networktraffic load and real packet delivery latency, can, in somesystem parameters conditions, affects the final performance,by giving remarkable difference in the results. Other intensivetests should be performed to better understand the maindifference in the final performance investigation. These trialsshould be performed also considering other routing protocolsfor ad hoc networks, such as AODV and DSR.

REFERENCES

[1] P.Bergamo, D.Maniezzo, A.Giovanardi, G.Mazzini, M.Zorzi, "Dis-tributed Power Control for Power-aware Energy-efficient Routing in AdHoc Networks," in Proc. of EW2002, Florence, Italy, Feb. 2002, pp.237-243.

[2] "OPNET commercial tool," http://www.opnet.com.[3] "The Network Simulator ns2," http://www.isi.edu/nsnam/ns/.[4] "GloMoSim," http://pcl.cs.ucla.edu/projects/glomosim/.[5] K. Fall, "Network emulation in the VINT/NS simulator," in Proc. of

IEEE Computers and Communications, 6-8 July 1999, pp. 244-250.[6] R.J. Wellington, M.D. Kubischta, "Wireless network emulation for dis-

tributed processing systems," in Proc. of IEEE Military CommunicationsConference, Oct. 2003, Volume: 1 , 13-16, pp. 475-480.

[7] J. P. Macker, W. Chao, J. W. Weston, "A low-cost, IP-based mobilenetwork emulator (MNE)", MILCOM 2003 - IEEE Military Communi-cations Conference, vol. 22, no. 1, Oct 2003 pp. 481-486.

[8] M. Matthes, H. Biehl, M. Lauer, 0. Drobnik, "MASSIVE: An EmulationEnvironment for Mobile Ad-Hoc Networks" in Proc. of Wireless On-demand Network Systems and Services, (WONS 2005), 19-21 Jan. 2005,pp. 54-59.

[9] P. Jacquet, P. Muhlethaler, A. Qayyum, "Optimized Link State RoutingProtocol," IETF Draft, February 2000, http://www.ietf.org/ID.html.

[10] A. Giovanardi, G.Mazzini, "Emulation Architecture for Ad Hoc Net-works", IEEE MEDHOC 2005, France.

[11] A. Giovanardi, G.Mazzini, "Network Emulation in the SAM Simulator",IEEE PIMRC 2005, Berlin, Germany.

[12] S. McCanne, C. Leres, V. Jacobson, "Libpcap," 1994.ftp:H/ftp.ee.lbl.gov/libpcap.tar.Z

[13] R. Stevens, "UNIX Network Programming, Volume 1: Networking APIs- Sockets and XTI," 1998, Prentice Hall PTR.

[14] RFC 1014, "XDR: External Data Representation standard,"http://www.faqs.org/rfcs/rfc 10l 4.html.

[15] Scripted Display Tool (SDT), http://proteantools.pf.itd.nrl.navy.mil/sdt.html

144

. . PATH LOSS, N=10, emul .PATH LOSS, N=15 emul -

..*. PATH LOSS, N=20 emul --*--

..

........ ... ._'---------------

-4 "' ., -".

"'S -, ".,,

-~~~~~~~

z