Download pdf - PaulMüller*,DennisSchwerdelandJustinCappos ......node into several virtual nodes. These virtual nodes have an increased parallelism since multiple instances can be run on one host,

Paul Müller*, Dennis Schwerdel and Justin Cappos

ToMaTo a Virtual Research Environment for LargeScale Distributed Systems Research

*Paul Müller: E-Mail: [email protected] Schwerdel: E-Mail: [email protected] Cappos: E-Mail: [email protected]

Abstract: Networks and distributed systems are an impor-tant field of research. To enable experimental research inthis field we propose a new tool ToMaTo (Topology Man-agement Tool) which was developed to support researchprojects within the BMBF funded project G-Lab. It shouldsupport researchers from various branches of science whoinvestigate distributed systems by providing a virtual en-vironment for their research.

Using various virtualization technologies, ToMaTo isable to provide realistic components that can run real-world software as well as lightweight components that canbe used to analyze algorithms at large scale.

This paper describes how an additional virtualizationtechnology from the Seattle testbed has been added toToMaTo to allow even larger experiments with distributedalgorithms. Moreover the paper describes some concreteexperiments that are carried out with ToMaTo.

1 Introduction

Many research projects focus on improving various aspectsof distributed networks e.g. networking concepts and ar-chitecture of networks in general [MR08]. All of theseresearch projects need ways to evaluate their ideas andresults. Simulations can model reality, but will never showunforeseen behavior as real systems do. Therefore re-search environments aim to provide a realistic environ-ment for experiments.

Realism is an important aspect of a virtualized envir-onment it describes how similar the environment is to thereal world. Some effects of reality are only visible inenvironments with a certain degree of realism, whichrestricts the kinds of experiments that can use the re-search environment. On the other hand it is important touse the resources efficiently and run as many experimentsas possible on as few resources as possible since researchenvironments are expensive in both acquisition and op-eration. There exists an obvious trade-off between paralle-

lism and realism in practice. Real systems offer the bestrealism possible but only allow one experiment to use theresources at a time. Research environments can use vir-tualization technologies to achieve parallelism at the costof losing realism.

Most existing research environments for distributedsystems (testbeds) operate on a fixed level of realism andparallelism. A lack of parallelism results in highly limitedresources so that only a small number of instances can beused across all researchers. Conversely, if an environmenthas a limited amount of realism, certain types of experi-ments cannot be run at all. Federations allow users tocreate experiments spanning testbeds with different de-grees of realism and parallelism. This reduces resourcewastage but also causes new problems since some testbedfeatures are not usable across testbed boundaries.

The topology management tool (ToMaTo) [SHG+11]has been developed as part of the German-Lab project[Ger, SGH+10] to provide a virtualized research environ-ment for networking experiments. ToMaTo uses a uniqueapproach to reduce resource wastage: Users can selectbetween multiple virtualization technologies for eachcomponent of their experiment. This way each componentuses only the resources it needs to provide the realism thatthe experiment needs and thus parallelism is maximizedand resource wastageminimized. ToMaTo effectively oper-ates on multiple levels on the realism/parallelism scale asshown in Figure 1.

Figure 1: Realism/Parallelism scale.

DOI 10.1515/pik-2013-0043 PIK 2014; aop

Bereitgestellt von | Universitaetsbibliothek KaiserslauternAngemeldet | [email protected] Autorenexemplar

Heruntergeladen am | 25.01.14 09:09

This paper describes how the restricted python language(Repy) [CDR+10] developed as part of the Seattle testbed[CBKA09] has been added to ToMaTo as a new virtualiza-tion technology that offers extremely lightweight program-mable devices. Section 2 analyses related work and discussvirtualization technology used by different testbeds. TheToMaTo testbed is described in Section 3 and Section 4explains the integration of the Repy technology. Section 5explains the usage of ToMaTo in experimentation and sec-tion 6 explains three experiments that have been con-ducted using ToMaTo before section 7 concludes.

2 Related work

Emulab [GRL05, ESL07] is a testbed that does not usevirtualization technology for its hosts, so physical compu-ters can be used for experiments. While this setup offersthe best possible realism, it maywaste resources since onlyone experiment component can be run on a host at a time.To mitigate this, Emulab also provides a small number ofnodes that have operating system virtualization and mayact as multiple different nodes. This turns one physicalnode into several virtual nodes. These virtual nodes havean increased parallelism since multiple instances can berun on one host, but the hosts are still exclusively used byone experiment. The virtual nodes also have reduced rea-lismwhen compared to physical nodes.

The Planet-Lab testbed [PBFM06, PR06] uses contain-er-based virtualization technology and offers its users ac-cess to Linux hosts with limited kernel-space access. Thisvirtualization technology is a good compromise of realismand parallelism from an operating system point of view.Still, some experiments can not be executed if they needmore access to the operating system or, more likely, thenetwork configuration. The parallelism is much greaterthan dedicated environments like Emulab offers. However,resources are still wasted since the container-based virtua-lization runs one complete user space system for eachvirtual device.

The Seattle testbed [CBKA09] allows its users toexecute Repy programs on the testbed nodes. Testbednodes are actually multi-use devices like laptops, phones,desktops, etc. that are used by end users around theworld. Repy programs can use limitedmemory and proces-sor resources on the host computer, and network resourcesare shared by assigning different port ranges to differentRepy programs. The Repy language and the restrictionsenforced on Repy programs hugely limit the realism of theSeattle testbed. On the other hand, the Seattle testbedoffers a very high parallelism. There are only few dedicated

Seattle servers as the parallelism would be limited byport number restrictions, in fact most Seattle resources areobtained by volunteer computing.

Simulation tools like ns-3 [ns311], omnet [VH08] andmininet [LHM10] model reality to a certain degree and areable to run huge experiments. Simulations can modify theprogress of time in the simulated environment and thus usethe full resources of the host during the whole experimentrun. Therefore simulators offer the best possible resourceusage but also a very limited realism since the simulationenvironment contains only amodel of the reality.

3 ToMaTo

ToMaTo is a topology-oriented networking testbed de-signed for high resource efficiency, i.e. high parallelismwhere possible but high realism where needed. Topologiesconsist of devices (produce and consume networking data)and connectors (manipulate and forward data). Devicescontaina set of networking interfaces that canbeconnectedto connectors. Figure 2 shows a simple topology consistingof five devices (one central server and four clients) andthree connectors (two switches and one Internet connec-tor). To increase both flexibility and resource efficiency,

ToMaTo offers different types of devices and connec-tors. Users can choose between hardware virtualization,which provides an environment nearly identical to a realcomputer but has a high resource usage, and containervirtualization that uses fewer resources but does not suitall needs. The topology in Figure 2 uses hardware virtuali-zation for the server since it runs a software that requireshardware virtualization. To save resources it uses contain-er virtualization for the clients. This example shows thatthe testbed offers full flexibility on the one hand while alsosaving resources on the other hand.

Figure 2: Example topology.

The default connector type is a VPN connector with aselectable forwarding policy (hub, switch or router). Public

2 Paul Müller, Dennis Schwerdel and Justin Cappos



services, cloud resources or even other testbeds can becombined with ToMaTo topologies by using external net-work connectors. To help users with running their net-working experiments, ToMaTo offers an easy to use, web-based front-end with an intuitive editor. Users can controltheir devices directly from their browser or using a VNCclient of their choice. Advanced tools like link emulationand packet capturing are included in ToMaTo and can beused to run and control experiments.

Figure 3: ToMaTo structure.

The ToMaTo software consists of three tiers as shown inFigure 3. The hosts provide virtualization technology anda complete toolset needed for advanced features like linkemulation, packet capturing, etc. The back-end compo-nent contains all the control logic of the ToMaTo softwareand remotely controls the hosts. Different front-ends usethe XML-RPC interface provided by the backend compo-nent. The most important front-end is the web-based userfront-end that allows users to edit and manage their topol-ogies from their browser using modern web technologies.Other front-end software includes a command-line clientthat allows easy scripting and an adapter for the Teaglefederation framework [CMW10, WMFP11].

Figure 4: ToMaTo editor.

One of the key features of ToMaTo is its graphical userinterface that allows even unexperienced users to createcomplex network topologies by drag and drop. ToMaTo

features an easy-to-use editor (Figure 4) to create, manage,and control networking topologies. With this editor, userscan configure their topology components and control themin the same intuitive interface. The editor also gives usersdirect access to their virtual machines using web-basedVNC technology.

4 Repy

The Repy sandbox was created to isolate code for theSeattle testbed [CBKA09]. The major goals of the sandboxare to provide performance isolation, security isolation[CDR+10], and a portable programming interface acrossdiverse device types. At a high level, a Repy sandbox isintended to be secure enough that one would allow a nonexperienced user to execute code within it. In this way, it issomewhat similar to existing sandboxes like Flash andJava, except these are meant to execute code from visitedwebpages and the like. The Seattle model intends thatresources will be used in more of a cloud like manner, sothat unknown parties may push code to a user’s machineon demand. However, the attack surface is minimized, inpart by restricting the sandbox functionality.

Clearly, performance isolation is also an importantgoal for Repy. A Repy sandbox is meant to be lightweightyet have the same sort of performance isolation that existsin commercial operating system VM offerings. To achievethis, Repy uses operating system hooks commonly usedfor debugging to vary the amount of CPU and memoryavailable to a sandboxed program. To restrict other re-sources such as network and disk I/O, Repy checks callsthat access these resources and prevents or delays theirexecution if they go over a configurable quota. This allowsthe performance impact of a Repy program to be selectedand controlled. (For our purpose, this allows the amountof parallelism to be very high while retaining realisticperformance isolation amongst instances.)

The Repy sandbox is based upon Python and con-sumes few resources. Running a hello-world Repy programtakes about 1 MB of memory on most computers. The per-formance impact seen by running code in Repy dependson the operation, but averages about 7% over native Py-thon onmost benchmarks.

While Repy’s interface was designed explicitly tomini-mize and secure the attack surface, it was also designed forextensibility in two ways. First, a mechanism similar tosystemcall interposition (or object capability systems) isbuilt into Repy. This allows one to enforce policies upon aRepy sandbox without needing to change the trusted com-puting base [CDR+10]. For example, if one wants to restrict

ToMaTo a Virtual Research Environment for Large Scale Distributed Systems Research 3



access to certain network destinations or sources, this canbe added in a securemanner without changing Repy.

Repy itself was also designed to be extendable byadding new functionalities. Adding access to a new type ofdevice or changing the level of access to a device requiresa few minor modifications to the code. One simply codesthe new routines, connects them into a module that per-forms a security validation on data passing through it, andthis is now available to Repy programs running in thismodified sandbox. This was leveraged to integrate Repywith ToMaTo.

4.1 Integration

ToMaTo’s structure and its support for different device andconnector types make it very flexible and extensible. Thebasic interaction point that all devices and connectorsmust support are network bridges where virtual networkinterfaces can be connected. New element types can beeasily added to ToMaTo as long as they fit into the basictopology model (devices with interfaces, connectors andconnections) and support bridges as interaction points. Allother features are optional and can be described by cap-ability records. Thus, the integration of programmabledevices as a new device type was easily possible withminimal changes to ToMaTo.

The first part of the integration of Repy as programma-ble device into ToMaTo was adapting Repy and thus creat-ing ToMaTo-Repy. In the Seattle testbed, the interface forRepy scripts contained methods to access the network viaUDP/IP and TCP/IP, to read and write files in a dedicatedfolder and some utility methods to retrieve the time, settimers etc. These methods needed to be exchanged withnetwork interfaces speaking Ethernet tomatch the ToMaTomodel.

ToMaTo-Repy extends Repy and adds the capability tocreate and connect to virtual network interfaces (VNIs).The names of the VNIs and their alias seen by Repy-scriptscan be set via command-line arguments. The programminginterface exposed to the repy-scripts has been extended toinclude methods to list available VNIs, send data on aspecific VNI and receive data from one or all VNIs. Themethod to receive data allows non-blocking calls as wellas blocking calls with a configurable maximum timeout.To avoid polling, a special method is exposed that listenson all VNIs at the same time and returns the received dataas well as the name of the VNI it has been received on.

The data units that scripts in ToMaTo-Repy work withare raw Ethernet packets. The scripts are free to parse theheaders according to Internet standards or to use custom

protocols to speak to other devices. Also the networkingbehavior (i.e. protocol handling) can be completely de-fined by the scripts, and there is no kernel that will processthe Ethernet packets. Despite the name “programmabledevice”, users can also use these devices to implementconnectors andmiddleboxes like firewalls, routers, etc.

To simplify the usage of ToMaTo-Repy when workingwith Internet protocols, the struct library has been addedto the interface for scripts to help encoding and decodingbinary data structures. Also a collection of ToMaTo-Repyscripts containing implementations of several Internet pro-tocols is publicly available and constantly extended tohelp users program their programmable devices.

ToMaTo-Repy has been packaged for Debian-basedsystems like the ToMaTo hosts and is installed during setupof the hosts. The backend component of ToMaTo has beenextended to include ToMaTo-Repy as a third device typecalled programmable device. Users can configure para-meters that are passed to the script upon execution. Thisway the same script can be used for multiple devices andvariable data like addresses can be configured separately.The virtual machine based devices allow users to uploadand download the device image, i.e. the virtual harddisk.For programmable devices this paradigm is used to uploadand download the ToMaTo-Repy scripts to/from the de-vices. Also a collection of ready-to-use images, called tem-plates, is available for programmable devices like it is forthe other device types. These templates include a pingablenode, a switch, a DNS server, and a DHCP server. Theoutput of the device’s script is directed to a log file and canbe viewed using the same VNC technology used to accessthe consoles of virtualmachine based devices.

4.2 Evaluation

The introduction of programmable devices into ToMaTousing Repy from the Seattle testbed allows users to usenetworking devices in their experiments that can be easilyprogrammed but do not have the resource usage and over-head of virtual machines.

ToMaTo-Repy offers users to program protocols atEthernet level with the simplicity of the high-level pro-graming language python. The library “tomatolib” con-tains a collection of protocol implementations and codesnippets written in ToMaTo-Repy that can be used to writecomplex programs. Table 1 shows the code sizes of exam-ple programs by counting non-empty lines of code of thescript Lines of code Script Library Total defining the pro-gram, the included parts of tomatolib and the total lines ofthe combined script.




Table 1: Source code size examples.

Besides simplicity, resource usage and performance areimportant aspects of the new device type. The resourceusage of the programmable devices depends on the scriptthat is being executed. For most scripts, the overhead ofthe python interpreter and the Repy sandbox is muchmorethan the resources needed by the script itself. The amountof RAM that is being used has been measured to be be-tween 5 and 10 megabytes, but of course this value de-pends on the software versions and the script. The Repysandbox has the ability to restrict the resource usage of thescripts it executes, which could be used to limit the re-source usage when needed.

Since Repy runs in user space, additional contextswitches are needed for packet processing. Also python isexpected to perform worse than the highly optimized ker-nel code written in C. A ping experiment has been run todetermine the additional overhead of a Repy node com-pared to pinging interfaces of the virtual machine baseddevices. Two different Repy nodes have been implementedthat both are able to respond to ARP requests and to ICMPecho requests. The first implementation hierarchically dis-sects the packet headers and processes them in a modularway while the second implementation picks only thosebyte ranges from specific offsets that are needed to detectthe request and recombines them to form the answer.Figure 5 shows the CDF and themedian RTT of the differentimplementations and device types. Repy performs betterthan the KVM hardware virtualization since the Repy scriptdoes not have the overhead of a complete kernel. TheOpenVZ container virtualization is able to beat all otherdevice types because it handles the pings inside a differentnamespace of the host kernel without any contextswitches. Figure 5 shows, that the Repy programmabledevice introduces an overhead between 0.1 and 0.15 milli-seconds due to context switching and the python lan-guage. Notably the optimized Repy implementation wasable to reduce the overhead significantly. Even for thenormal implementation, the overhead introduced is lowenough to allow networking experiments with precise linkemulation. Since version 2.1, the programmable device isan integral part of ToMaTo and users can use ToMaToRepyscripts to build complex yet resource efficient devices.Experiments have shown that the number of programma-ble devices that can run concurrently on one host is at least

1000 but these values depend on the activity and complex-ity of the scripts. ToMaTo-Repy helps to extend the rea-lism/parallelism range of ToMaTo and thus improve itsflexibility and efficiency.

Figure 5: CDF of RTT experiment.

5 Experimentation

Evaluating distributed algorithms is a complex task. Oneapproach is to compare the design goals or requirementswith the actual capabilities of the resulting software. Incase of an experimental facility the design goal is to sup-port experiments and help researchers carry out theirexperiments. To evaluate ToMaTo based on this goal sec-tion 5.1 first develops a classification of experiments inthe German-Lab project and section 5.2 outlines how To-MaTo supports these types of experiments. Section 5.3takes a quick look at the efficiency and scalability ofToMaTo.

5.1 Types of experiments

In general four different experiment types can be identifiedin networked respectively distributed systems research.

Access layer experiments. Access layer experiments consid-er the lower networking layers and examine the usage ofhardware for networking. An example for this experimentclass is mobile handover protocols. These experimentsneed access to real hardware, they often need to run cus-tom operating systems (e.g. with real-time support) andthey need heterogeneous access technologies (3G, IEEE802.11, Fiber, etc.). In most cases, these requirements can




only be fulfilled with custom testbeds, thus supporting thiskind of experiment was not a design goal for ToMaTo.

Network layer experiments. These types of experimentsconsider the TCP/IP suite and its networking layers. Exam-ples for this class are experiments with IPv6 extensionsand TCP substitutes. This kind of experiment needs to runmodified kernels. The resources that a single experimentneeds are normally limited to a few devices but thesedevices have to be connected in complex network topolo-gies with link emulation.

Protocol/Algorithm experiments. Experiments with proto-cols or algorithms work on top of the network layer andusually consider new approaches for lager networks.Nearly all peer-to-peer experiments fall in this category.These experiments need a high number of nodes but usual-ly no hardware or kernel access. They only need simplenetwork topologies with link emulation.

Legacy application experiments. Experiments using legacysoftware cannot be modeled because of its unspecified orunpublished behavior. Examples of this software are Skypeand Windows. The experiments with this software oftenneed special operating system environments including In-ternetaccessand linkemulation. In turn, theseexperimentsnormallydonotneedbigor complexnetwork topologies.

Experiences of the German-Lab experimental facility showthat most experiments can be categorized fairly wellwith this scheme. A few experiments have two experimentclasses, and thus have requirements of both classes. Theresource requirements of the classes are very heteroge-neous but a general tradeoff betweenmore resource accessand access to more resources becomes evident, i.e. in gen-eral experiments either need a high number of (light-weight) resources with restricted access or a small numberof resources with full access.

5.2 Experiment support in ToMaTo

ToMaTo has been designed to support all experimentclasses identified in section 5.1 except for access layerexperiments because these experiments need a specializedtestbed depending on the access technology. The Wisebed[BCD+10] and DES testbed [Des10] for example are specia-lized experimental facilities for sensor networks and wire-less networks.

Network layer experiments canbedone easily in ToMa-To using KVM devices and virtual switches. The KVM de-

vices over all need flexibility in kernel choice andmodifica-tion required by this experiment class. Switched networksprovide connectivity on layer 2 so that anyTCP/IPmodifica-tion or substitute can be examined. Using the commandline frontend even very complex topologies can be easilydesigned. The possibility to capture anddownloadnetworktraffic canbe veryhandy for this kindof experiment.

Protocol/Algorithm experiments are supported in To-MaTo using OpenVZ devices. Since OpenVZ devices arevery lightweight, a high number of devices can be usedin topologies. Using an Internet connector, external re-sources like PlanetLab nodes can be included in the ex-periment. Using the upload/download image feature,users can prepare a device image once and upload it to allof their devices. Capturing network traffic and link emula-tion can be used to debug the protocols.

ToMaTo also supports legacy application experimentsusing KVM devices and Internet connectors. KVM devicescan run nearly all x86 operating systems including Win-dows and BSD, therefore users can build custom environ-ments for their legacy applications. The legacy applicationcan communicate with external services using the Internetconnector. Traffic of the legacy application can be cap-tured and analyzed using specialized tools without anyoperating system support.

5.3 Efficiency and scalability

With ToMaTo, users can choose between OpenVZ andKVM virtual machines. This way, users can get the level ofaccess that is needed for their experiments and still use asfew resources as possible. A single cluster node can handleup to 250 OpenVZ devices and up to 50 KVM devices, bothdepending on device usage. The networking componentsonly pose a very small overhead and can handle connec-tions with over 100Mb/s without influencing them.

ToMaTo hosts use an existing operating system asbasis and only need small changes that have beenbundled as a software package. This has the advantagethat operating system support and security updates areavailable from the original sources and do not have to beprovided by the experimental facility administrators. Asthe ToMaTo back-end only controls the hosts and onlycontacts them when users change their topologies, theback-end can handle many host nodes making the solu-tion very scalable.

ToMaTo can be used to create experimental facilitieswith distributed hosts. Limitations in network emulationapply since the resulting link characteristics are a combi-nation of real and emulated link properties. ToMaTo offers




long-term link statistics so the users can plan their experi-ments accordingly. ToMaTo allows its users to design,manage and control networking topologies for use in net-work research. ToMaTo fits for a wide range of experi-ments identified as common in the context of the German-Lab project. The design of the testbed software offersefficiency and scalability. ToMaTo is not bound to Ger-man-Lab and can easily be used to build similar experi-mental facilities.

In the German-Lab testbed currently 35 of 182 hostsare ToMaTo-enabled. The goal is to increase this numberalso by integrating nodes from international projects andthereby increase the usability of the testbed. Also theintegration of OpenFlow is already implemented.

6 Experiments in ToMaTo

6.1 Experiment: German-Lab Deep

ToMaTo has been successfully used for an experiment inthe context of German-Lab Deep [KVS+11] project thatfocuses on attack mitigation in VoIP systems. In this sce-nario, an attacker initiates malicious calls that must bedetected by the distributed VoIP system and blocked at thenearest gateway using cross-layer components. Figure 6shows the topology used for this experiment. It containsseveral end systems and smart middle boxes using theKVM technology. In this scenario, it was very important toexplicitly define the links in the topology and to be able to

route traffic over programmablemiddle boxes. This experi-ment was central to the whole project and brought togetherdifferent software components. A demonstration at theEuroview conference successfully showed the interoper-ability of these components using ToMaTo [KVS+11].

6.2 iGreenMobile Application Test

The research project iGreen aims to introduce location-based semantic services in the agricultural industry inorder to make use of data available in the public or privatesectors directly in the field. Using Internet-based servicesin rural areas is a challenging task because of the lack ofhighquality network coverage forWiFi or even cell phones.The use of satellite connections with high delay as well asthe low availability of network coverage for cell phonesimposes interesting requirements for testing mobile appli-cations used in the agricultural sector.

The iGreen mobile application (shown in Figure 7) isdesigned to offer access to various iGreen services support-ing the decision making process related to agriculturaltasks like proper fertilization or application of pesticides.This application will run on mobile devices in the field oras embedded software on farm machines. In both cases,the application has to access services that run in a com-pute cloud usingwhatever network connection is availableat its location, i.e., connectivity by a mobile carrier or by adirect satellite link.

Figure 7: The iGreenmobile application.

Figure 6: The topology used by a DeepG experiment. It contains anattacker, a victim and several components needed tomitigate thisattack. Smart routers connect all components.




In an experiment using ToMaTo [SGR12], the prototypeversion of the mobile iGreen application has been testedin different network connectivity scenarios to analyze theinfluence of delay and packet loss on the service quality.In this scenario, the mobile application runs in an emu-lator, embedded in a ToMaTo topology that contains aconfigurable link and a bridge to the Internet where theiGreen service resides. The test has been carried out withdifferent link properties, matching the characteristics ofUMTS, GPRS and Satellite links. Hence different imple-mentation problems resulting in a degraded service qual-ity have been identified. For example, the software re-trieved the messages by first requesting a list of messageIDs and then requesting each message synchronously ina loop. This resulted in huge delays due to the accumu-lated round-trip-times that could be reduced by request-ing all messages asynchronously in parallel.

6.3 Malware Analysis Experiment

A live demonstration at the Euroview conference 2011[SRM11] showed the unique features of ToMaTo that allowusers to analyze network behavior of malicious software.In this demonstration, the network behavior of an olderInternet worm has been analyzed using ToMaTo. Figure 8shows the topology that has been used in the demonstra-tion. It clearly shows that the Internet in the upper leftcorner is not connected to the rest of the topology, so allthe other components of the topology had no access tothe Internet and the worm could safely be analyzed with-out the risk of escape. In most other networking testbeds,this would not be possible, because there is an implicitnetwork link to the Internet or to core components of thetestbed.

Figure 8: The topology used for a malware analysis experiment. Thistopology is not connected to the Internet for safety reason.

In the analysis of the malware, first a virtual machineusing KVM and running Windows XP has been configured.

A backup of that machine has been downloaded to be ableto replay the infection later. Using the packet-capturingfeature on the outgoing link and scripted network compo-nents it was possible to analyze the network behavior ofthat software without relying on tools that run inside theinfectedmachine.

As a result of the analysis as shown in Figure 9, thecontrol server has been identified as a probably hackedname server. The protocol used by the control server couldbe identified as IRC and even the channel could be cap-tured.With this information it would be possible to contactthe povider of the control server and shut it down or to trya man-in-the-middle attack on the control protocol forfurther analysis.

Figure9:The resultsof themalwareanalysis reveal thatahackednameserver is used as amaster server that controls the victims using IRC.

7 Conclusion

To conclude, this paper describes how the Repy sandboxfrom the Seattle project was integrated with ToMaTo. Thisallows ToMaTo to run experiments that have a high degreeof parallelism, while reducing the realism. By combiningRepy together with the other virtualization technology,ToMaTo can obtain a much wider range of parallelism/realism.

This result shows how well-suited ToMaTo is for dis-tributed systems research since it can provide realisticcomponents that can run real-world software as well aslightweight components that can be used to analyze algo-rithms at large scale. This flexibility allows ToMaTo to usemuch less resources than other comparable testbeds inmost scenarios.

Since the research projects based on German-Labended, the GLab facility including the ToMaTo testbed isopen for external users. Interested researchers can contactthe authors or visit the webpage at http://www.tomato-lab.org for information on accessing the testbed.

The integration of the Repy technology shows howeasy it is to integrate new technology into ToMaTo be-




cause of its flexible architecture. ToMaTo developers arealways open for external contributions and joint researchprojects aiming towards further ToMaTo developmentand usage.

References

[BCM+10] Tobias Baumgartner, Ioannis Chatzigiannakis, MaickDanckwardt, Christos Koninis, Alexander Kröller, GeorgiosMylonas, Dennis Pfisterer, and Barry Porter. Virtualisingtestbeds to support large-scale reconfigurable experimen-tal facilities. In Proceedings of EWSN – 7th EuropeanConference of Wireless Sensor Networks, pages 210–223,2010.

[CBKA09] Justin Cappos, Ivan Beschastnikh, Arvind Krishnamurthy,and Tom Anderson. Seattle: a platform for educationalcloud computing. In Proceedings of the 40th SIGCSE Techni-cal Symposium on Computer Science Education, SIGCSE2009, pages 111–115, 2009.

[CDR+10] Justin Cappos, Armon Dadgar, Jeff Rasley, Justin Samuel,Ivan Beschastnikh, Cosmin Barsan, Arvind Krishnamurthy,and Thomas E. Anderson. Retaining sandbox containmentdespite bugs in privilegedmemory-safe code. In ACM Con-ference on Computer and Communications Security,pages 212–223, 2010.

[CMW10] K. Campowsky, T. Magedanz, and S. Wahle. Resourceman-agement in large scale experimental facilities. InNetworkOperations andManagement Symposium (NOMS), 2010IEEE, pages 930–933, april 2010.

[Des10] DES-Testbed AWirelessMulti-Hop Network Testbed for fu-ture mobile networks, Stuttgart, Germany, 06/2010 2010.

[ESL07] Eric Eide, Leigh Stoller, and Jay Lepreau. An experimenta-tion workbench for replayable networking research. InProceedings of the 4th USENIX conference on Networkedsystems design & implementation, NSDI’07, pages 16–16,Berkeley, CA, USA, 2007. USENIX Association.

[Ger] German-Lab Project. German-LabWebsite <http://www.german-lab.de>.

[GRL05] Shashi Guruprasad, Robert Ricci, and Jay Lepreau. Inte-grated Network Experimentation using Simulation andEmulation. In Proceedings of the First InternationalConference on Testbeds and Research Infrastructures forthe DEvelopment of NeTworks and COMmunities,pages 204–212, Washington, DC, USA, 2005. IEEE Compu-ter Society.

[KVS+11] MichaelKleis, ChristianVaras, AbbasSiddiqui, PaulMüller,IrfanSimsek,MartinBecke,DirkHoffstadt, AlexanderMar-old, ErwinRathgeb,ChristianHenke, JuliusMüller, andTho-masMagedanz. Cross-layer security and functional compo-sition for a future internet. Proceedingsof 11thWürzburgWorkshopon IP: Joint ITGandEuro-NFWorkshop “VisionsofFutureGenerationNetworks” (EuroView2011), 2011.

[LHM10] Bob Lantz, Brandon Heller, and NickMcKeown. A networkin a laptop: rapid prototyping for software-defined net-works. In Proceedings of the Ninth ACMSIGCOMMWork-shop on Hot Topics in Networks, Hotnets ’10, pages 19:1–19:6, New York, NY, USA, 2010. ACM.

[ns311] NS3 The ns-3 network simulator. http://www.nsnam.org/,Accessed 2011.

[PBFM06] Larry L. Peterson, Andy C. Bavier, Marc E. Fiuczynski, andSteveMuir. Experiences Building PlanetLab. InOSDI, pages351–366, 2006.

[PR06] Larry L. Peterson and Timothy Roscoe. The design princi-ples of PlanetLab.Operating Systems Review, 40(1): 11–16,2006.

[MR08] Bernd Reuther and Paul Mueller. Future Internet Architec-ture – A Service Oriented Approach. it – Information Tech-nology, 50(6), 2008.

[SGH+10] Dennis Schwerdel, Daniel Günther, Robert Henjes, BerndReuther, and Paul Müller. German-Lab Experimental Facil-ity. In Proceedings of FIS 2010 – Third Future Internet Sym-posium, pages 1–10, 2010.

[SGR+12] Dennis Schwerdel, Joachim Götze, Bernd Reuther, andPaul Müller. Testing mobile apps in the tomato testbed.Proceedings of 12th Wuerzburg Workshop on IP (Euro-View), 2012.

[SHG+11] Dennis Schwerdel, David Hock, Daniel Günther, Bernd Reut-her, Paul Müller, and Phuoc Tran-Gia. ToMaTo – a networkexperimentation tool. In 7th International ICST Conferenceon Testbeds and Research Infrastructures for the Develop-ment of Networks and Communities (TridentCom 2011),Shanghai, China, April 2011.

[SRM11] Dennis Schwerdel, Bernd Reuther, and Paul Müller. Mal-ware analysis in the tomato testbed. Proceedings of 11thWuerzburgWorkshop on I (EuroView), 2011.

[VH08] András Varga and Rudolf Hornig. An overview of theOMNeT++ simulation environment. In Simutools ’08:Proceedings of the 1st international conference on Simula-tion tools and techniques for communications, networksand systems & workshops, pages 1–10, ICST, Brussels,Belgium, Belgium, 2008. ICST (Institute for ComputerSciences, Social-Informatics and TelecommunicationsEngineering).

[WMFP11] S. Wahle, T. Magedanz, S. Fox, and E. Power. Heterogeousresource description andmanagement in generic resourcefederation frameworks. In Integrated NetworkManagement(IM), 2011 IFIP/IEEE International Symposium on,pages 1196–1199, may 2011.

PaulMüller: University Kaiserslautern,Computer Science Department,Paul Ehrlich Street Bld. 34,67663 Kaiserslautern/Germany




http://www.german-lab.de

http://www.german-lab.de

http://www.nsnam.org/

Dennis Schwerdel:University Kaisers-lautern, Computer Science Department,Paul Ehrlich Street Bld. 34,67663 Kaiserslautern/Germany

Justin Cappos: Polytechnic Institute ofNew York University, SixMetroTech Center,Brooklyn, NY 11201, USA


