16
Future Generation Computer Systems 15 (1999) 659–674 Javelin: Parallel computing on the internet Michael O. Neary * , Bernd O. Christiansen, Peter Cappello, Klaus E. Schauser Department of Computer Science, University of California, Santa Barbara, CA 93106, USA Accepted 14 December 1998 Abstract Java offers the basic infrastructure needed to integrate computers connected to the Internet into a seamless distributed computational resource: an infrastructure for running coarse-grained parallel applications on numerous, anonymous machines. First, we sketch such a resource’s essential technical properties. Then, we present a prototype of Javelin, an infrastructure for global computing. The system is based on Internet software that is interoperable, increasingly secure, and ubiquitous: Java-enabled Web technology. Ease of participation is seen as a key property for such a resource to realize the vision of a multiprocessing environment comprising thousands of computers. Javelin’s architecture and implementation require participants to have access to only a Java-enabled Web browser. Experimental results are given in the form of a Mersenne Prime application and a ray-tracing application that run on a heterogeneous network of several parallel machines, workstations, and PCs. Two key areas of current research, fault-tolerance and scalability, are subsequently explored briefly. ©1999 Elsevier Science B.V. All rights reserved. Keywords: Distributed computing; High performance computing; Java; Internet; World Wide Web 1. Introduction We want to solve computational problems that can- not be solved on existing NOWs or supercomputers – that simply cannot be solved now. Imagine that you have a huge computational problem that you want to solve. You have an ‘embarrassingly’ parallel algo- rithm for it. However, it is too large to solve on a network of workstations or a supercomputer. You do not want to wait until networks of workstations get big enough, or until processors and/or supercomput- ers get fast enough. What do you do? Our vision is to harness the combined resources of millions of com- * Corresponding author. E-mail addresses: [email protected] (M.O. Neary), [email protected] (B.O. Christiansen), [email protected] (P. Cappello), [email protected] (K.E. Schauser) puters connected to the Internet, forming a power- ful heterogeneous computing environment for running coarse-grain parallel applications. We believe that the time is ripe for this next step. First, global computing has made impressive strides recently. For example, on June 17th, 1997, 56-bit DES [1] was cracked by dis- tributed.net, using approximately 78,000 computers, as many as 14,000 per day. The press release [2] states: This project demonstrates the kind of supercomput- ing power that can be harnessed on the Internet using nothing but ‘spare’ CPU time. “Imagine what might be possible using millions of computers con- nected to the Internet!” Aside from cryptography and other obvious mathematical uses, supercom- puters are used in many fields of science. “Perhaps a cure for cancer is lurking on the Internet?”, said Verser, “Or perhaps the Internet will become Everyman’s supercomputer.” 0167-739X/99/$ – see front matter ©1999 Elsevier Science B.V. All rights reserved. PII:S0167-739X(99)00017-5

Javelin: Parallel computing on the internet

  • Upload
    ucsb

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Future Generation Computer Systems 15 (1999) 659–674

Javelin: Parallel computing on the internet

Michael O. Neary∗, Bernd O. Christiansen, Peter Cappello, Klaus E. SchauserDepartment of Computer Science, University of California, Santa Barbara, CA 93106, USA

Accepted 14 December 1998

Abstract

Java offers the basic infrastructure needed to integrate computers connected to the Internet into a seamless distributedcomputational resource: an infrastructure for running coarse-grained parallel applications on numerous, anonymous machines.First, we sketch such a resource’s essential technical properties. Then, we present a prototype ofJavelin, an infrastructurefor global computing. The system is based on Internet software that is interoperable, increasingly secure, and ubiquitous:Java-enabled Web technology. Ease of participation is seen as a key property for such a resource to realize the visionof a multiprocessing environment comprising thousands of computers. Javelin’s architecture and implementation requireparticipants to have access to only a Java-enabled Web browser. Experimental results are given in the form of a MersennePrime application and a ray-tracing application that run on a heterogeneous network of several parallel machines, workstations,and PCs. Two key areas of current research, fault-tolerance and scalability, are subsequently explored briefly. ©1999 ElsevierScience B.V. All rights reserved.

Keywords:Distributed computing; High performance computing; Java; Internet; World Wide Web

1. Introduction

We want to solve computational problems that can-not be solved on existing NOWs or supercomputers –that simply cannot be solved now. Imagine that youhave a huge computational problem that you wantto solve. You have an ‘embarrassingly’ parallel algo-rithm for it. However, it is too large to solve on anetwork of workstations or a supercomputer. You donot want to wait until networks of workstations getbig enough, or until processors and/or supercomput-ers get fast enough. What do you do? Our vision is toharness the combined resources of millions of com-

∗ Corresponding author.E-mail addresses: [email protected] (M.O. Neary),[email protected] (B.O. Christiansen), [email protected] (P.Cappello), [email protected] (K.E. Schauser)

puters connected to the Internet, forming a power-ful heterogeneous computing environment for runningcoarse-grain parallel applications. We believe that thetime is ripe for this next step. First, global computinghas made impressive strides recently. For example, onJune 17th, 1997, 56-bit DES [1] was cracked bydis-tributed.net, using approximately 78,000 computers,as many as 14,000 per day. The press release [2] states:

This project demonstrates the kind of supercomput-ing power that can be harnessed on the Internetusing nothing but ‘spare’ CPU time. “Imagine whatmight be possible using millions of computers con-nected to the Internet!” Aside from cryptographyand other obvious mathematical uses, supercom-puters are used in many fields of science. “Perhapsa cure for cancer is lurking on the Internet?”, saidVerser, “Or perhaps the Internet will becomeEveryman’s supercomputer.”

0167-739X/99/$ – see front matter ©1999 Elsevier Science B.V. All rights reserved.PII: S0167-739X(99)00017-5

660 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

Clearly, large numbers of users are willing to par-ticipate in the global computation of suitable applica-tions. While dramatic and encouraging, this exampleshows that, of the tens of millions of computers con-nected to the Internet, only a small percentage actuallyparticipate. There are several reasons for this. No in-centive is provided beyond either curiosity, helping aworthy cause, or fame (if one is lucky enough to findthe key). Security, specifically lack thereof, is a majorinhibitor. Prospective participants must trust thepro-gram serverto serve them a program that does notcontain bugs or viruses that destroy or spy on localdata. Secondly, participating in global computing isadministratively complex for both the program serverand the participants. Global computing applicationsrun on different architectures and operating systems,requiring many executable versions of the application,as well as different downloading mechanisms. We be-lieve that lack of security, lack of potent tools fordeveloping and deploying global computing applica-tions, and lack of incentive all diminish the number ofactual participants.

The advent of a safe and portable language sys-tem such as Java has provided tools and mechanismsto address some of these issues and interconnectcomputer systems around the globe in cooperativework. We progress towards our vision by developinga flexible, easily-installed infrastructure for runningcoarse-grained distributed applications on any com-puter connected to the Internet.

In Section 2, we enumerate some fundamental is-sues in global computing. Then, we review the Javelinvision, as well as the existing Javelin prototype. Fol-lowing that, we discuss some research issues of high-est priority: fault-tolerance and scalability. Then, webriefly review the existing research efforts towards aninfrastructure for global computing, concluding thatthere is a basic need for research on fault-toleranceand scalability in the context of global computing.

2. Goals and issues

In this section we briefly describe some of the mostimportant research issues in the field of global com-puting. The list is by no means complete, since thisis a relatively new area and more topics are likely toarise in the near future. We have divided the issues

into three distinct groups. The first group contains sub-jects that are being addressed in the context of secure,portable languages like Java. Thus, we do not feel soconcerned about these issues since significant progresshas already been made.

The second group contains the topics that we con-sider the most important for our current research,whereas the third group contains issues that can bedealt with only after solving some of the problemsin the second group, and hence must be placed in amore distant future. The first group consists of thefollowing topics:

Ease of use: We view ease of use as a key prop-erty of the proposed infrastructure, since it relies onthe participation of thousands of users. We envision aweb-centricapproach where a user only needs ubiqui-tous software such as a Java-enabled web browser toparticipate.

Security: Executing an untrusted piece of code posesintegrity and security threats to the host. To protectthe host from buggy or malicious code, any untrustedprogram must be run within an environment that onlyallows limited access to system resources such as thefile system. Host security already has been addressedby a variety of mechanisms including Software FaultIsolation [3], Secure Remote Helper Applications [4],and interpreted languages such as Java.

Interoperability: In heterogeneous systems like theInternet, hosts and clients may have different instruc-tion sets, word sizes, or operating systems. The in-frastructure proposed must provide the means to over-come this heterogeneity. This issue has been addressedby either employing machine-independent languages,such as Java, E [5], and Limbo [6], or by providingmultiple binary executables [7]. Machine-independentlanguages achieve portability at the expense of someperformance; binary executables achieve performanceat the expense of portability. It is thus desirable to sup-port both approaches in order to meet the demands ofas many applications as possible.

Performance: Since our infrastructure aims at pro-viding better performance than is available locally,the efficient execution of anonymous code is essen-tial. The interpretation overhead of portable languagesis being overcome by modern compilation techniquessuch as just-in-time compilation [8,9] that allow foran execution speed close to that of compiled C or C++code.

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 661

The second group contains the four issues that arecurrently highest on our priority list. In particular, faulttolerance and scalability will be discussed in greaterdetail in Section 4.

Scalability: As performance relies heavily on thenumber of participants, scalability is a key issue. Weintend to provide an infrastructure that is scalable withrespect to communication (i.e., limitations imposed bysubsidiary technologies, such as Java applet security,must be overcome), computation (i.e., resource allo-cation must be distributed), and administration (i.e.,requiring neither login accounts on hosts nor operat-ing system changes).

Fault-tolerance: In a potentially Internet-wide dis-tributed system, fault-tolerance is crucial; hosts maybe untrusted and communication is unreliable. It canbe provided either transparently by the underlying ar-chitecture or explicitly by the application itself.

Incentive: To grow the infrastructure to thousandsof hosts, we need to give potential hosts an incentiveto participate. We can capture the interactions betweenclients and hosts in amicroeconomic modelwhichmodels the trading of computing resources. We feelthat the application of market mechanisms is the rightway of solving the problem of resource allocation inpotentially world-wide distributed systems. Importantunderlying technologies required for the implemen-tation of a market mechanism, such as cyber-money,authentication schemes, and secure transactions, arematuring rapidly.

Correctness: Economic incentives have a dark side:the specter of hosts faking computations and returningwrong or inaccurate results. To deal with this prob-lem, clients may be able to apply algorithm-specifictechniques that cheaply verify a computation’s cor-rectness (e.g., it is simple to verify a proposed solu-tion to a system of linear equations), or cheaply ver-ify with high probability the correctness of a com-putation. Other techniques include statistical methods(e.g., in an image rendering computation, the clientcan compute some random pixels and compare theseto the corresponding pixels returned by hosts), check-sum computations that are inserted into the code toassure that the code was actually run, or redundantcomputations (i.e., performing a computation by sev-eral hosts). Reputation services, similar to credit andbond ratings, also give hosts an economic disincentivefor faking computations.

Resource allocation using market-based mechanismis one of our more long-term goals in its full scale,although a simpler form of incentive – simply exploit-ing natural curiosity and the desire to get famous –is very much a part of our current research. Besidesthese, there are other important issues we might con-sider in the future but do not intend to address in thenear term:

Programming models: The envisioned infrastruc-ture provides a substrate on which various communi-cation, data, and programming models may be imple-mented; different models are suitable for different ap-plications. We intend to provide the programmer withabstractions, such as a global file system, shared mem-ory, and reliable communication channels.

Locality: At present, neither the latency nor band-width across the Internet are satisfactory for commu-nication intensive parallel applications. Thus, bothmust be taken into account when mapping applica-tions to the available resources. Ultimately, this im-plies the need for methods to determine or forecast thecommunication latency and bandwidth requirementsand execution time constraints for given problems[10].

Client privacy: A company’s internal data and‘know-how’ represent a value that usually is protectedfrom unauthorized access. The proposed infrastruc-ture must provide mechanisms that enable clients tohide the data and possibly the algorithms that arepassed to untrusted hosts (see e.g., Feigenbaum [11]).Although encrypted computing might not be possiblefor all applications, a number of important practi-cal problems can be encrypted [12]. Another way ofensuring client privacy is to split the computationinto fragments such that no part by itself reveals anyuseful information about the complete computation.

Quality of service: Quality of service must be in-corporated and ensured. A host negotiating for a taskshould be able to calculate accurate runtime estimatesbased on the task’sprofile as well as its own machinecharacteristics, in order to decide whether it can meetthe client’s runtime constraints without actually per-forming the computation. One possibility is throughmicro-benchmarks that characterize the host as a pointin a multidimensional performance space [13]. A suit-able benchmark must be representative, quickly com-putable by the host, and the broker must be able toquickly evaluate the result.

662 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

Fig. 1. The Javelin architecture.

In the following sections we will explain to whatextent we have already solved some of the given prob-lems and how we intend to attack others.

3. The Javelin infrastructure

In this section, we describeJavelin[14], our proto-type infrastructure for Internet-based parallel comput-ing using Java. Our system is based on Internet soft-ware technology that is essentially ubiquitous: Webtechnology. The current prototype already providessome of the properties listed in Section 2, like e.g.,ease of use and interoperability. Others, like scalabil-ity and fault-tolerance, will be addressed in the nearfuture (see Section 4).

Once these fundamental technological issues havebeen solved, we will be in a situation where compu-tation is truly a tradable commodity, and buying andselling of computational resources will become feasi-ble. At this point we will integrate market-based proto-cols and algorithms into our design, leading eventuallyto a system that is naturally load-balanced around theglobe and offers users a true incentive to participate.

The basic system architecture is shown in Fig. 1.There are three system entities – clients, brokers andhosts. A client is a process seeking computing re-sources; ahost is a process offering computing re-sources; abroker is a process that coordinates the al-location of computing resources. Clients register theirtasks to be run with their local broker; hosts registertheir intention to run tasks with the broker. The bro-ker assigns tasks to hosts that, then, run the tasks andsend results back to the clients. The role of a host or aclient is not fixed. A machine may serve as a Javelinhost when it is idle (e.g., during night hours), whilebeing a client when its owner wants additional com-puting resources.

3.1. Applications

One of the most important goals of our research isto find and classify new applications for our infras-tructure. We intend to further refine and optimize theapplications we have already implemented, as well asport some new applications that promise further in-sight into what is required to make the infrastructureas useful as possible.

3.1.1. Structure of an applicationBefore we list some actual applications, we need

to make a few assumptions about the type of appli-cation/programming model that will be supported bythe system:

Ideally, applications should beadaptively parallel,i.e., they should not demand a fixed number of hosts atstartup, should be able to change the number of hostsat runtime, and should generally be oblivious to thenumber of hosts at any given moment.

The primary programming model supported is aso-calleddynamic bag-of-work1 model, where ini-tially the client has a large task which can be splitinto smaller tasks and assigned to other hosts. Thesesmaller tasks can in turn be split and reassigned, and soon, until the granularity is so small that splitting doesnot make sense anymore. The task is then executed.It is important that each task be independent and alltasks be equally important to avoid prioritization.

For fault-tolerance purposes we assume all tasks tobe idempotent, i.e., after the first execution of a taskrepeated execution of the same task does not changethe result. For more details, see Section 4.

If an application does need to specify a fixed numberof processes, it should not expect its local broker toactivelystart finding the desired number of hosts fora one-to-one process to host mapping by contactingneighboring brokers, since this could easily flood thenetwork. Instead, the local broker may opt to schedulemore than one process to a single host and will decideimmediately or after a certain period of time if the hostdemand can be met. This restriction greatly simplifiesthe task of a broker and should be considered a startingpoint that can be modified in future.

1 We also like to call thisMaster-Worker with Subcontracting,and find this term slightly more intuitive.

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 663

As the most general parallel programming modelthe system will also supportmessage passingbased onsend and receive primitives. However, in this contextfault-tolerance will be limited since the system cannotknow about the semantics of the application, so it willnot know how to react to host and network failures onbehalf of the application.

3.1.2. Current applicationsWe have a pool of applications which vary in struc-

ture, complexity and difficulty of implementation. Thelist of applications we are currently considering con-tains:

Mersenne primality testing– this is our structurallyeasiest application. The computation to communica-tion ratio is huge for large primes. It fits both the mes-sage passing and the bag-of-work paradigm.

Image rendering– fitting the bag-of-work model,this application is an ideal starting point for experi-ments with various fault-tolerance methods (see Sec-tion 4).

Seismic data processing– traditionally a clustercomputing application fitting the message passingmodel, this is our most demanding application to date.Implementing it on top of Javelin requires the useof Java applications since local file I/O is required.However, this also means the development of a newhost security model for Javelin applications.

Beyond those applications listed, our goal is to findas many additional applications that can benefit fromthe proposed architecture as possible. In the near fu-ture, however, we intend to concentrate on those listedand demonstrate that the infrastructure can greatly im-prove their performance, opening up the possibility ofrunning applications much larger than anyone wouldhave deemed possible a few years ago.

3.2. Preliminary results

3.2.1. Javelin prototypeOur most important goal issimplicity, i.e., to en-

able everyone connected to the Internet or an intranetto easily participate in Javelin. To this end, our designis based on widely used components: Web browsersand the portable language Java. By simply pointingtheir browser to a known URL of a broker, users auto-matically make their resources available to host parts

Fig. 2. Steps involved in the remote execution of an applet.

of parallel computations. This is achieved by down-loading and executing an applet that spawns a smalldaemon thread that waits and ‘listens’ for tasks fromthe broker. The simplicity of this approach makes iteasy for a host to participate – all that is needed is aJava-capable Web browser and the URL of the broker.

Tasks are represented as applets that are embed-ded in HTML pages. This design decision impliescertain limitations due to Java applet security: e.g.,all communication must be routed through the brokerand every file access involves network communica-tion. Therefore, in general, coarse-grained applicationswith a high computation to communication ratio arewell suited to Javelin. Fig. 2 shows the steps involvedin the remote execution of an applet. These steps are:1. The client uploads the applet and an embedding

HTML page on an HTTP server. Clients runningtheir own HTTP server may skip the first step.

2. The client registers the corresponding URL witha broker.

3. The host registers with the broker its intention toexecute tasks, and retrieves the URL of a task.

4. The host downloads the HTML page from theHTTP server, and executes its embedded applet.

5. The host stores the result at the server site. If com-munication between cooperating hosts is required,messages are stored at, and retrieved from, theserver site.

6. The client retrieves the result.In the following we briefly discuss performance num-bers from our prototype. We conducted our perfor-

664 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

Fig. 3. Raytraced images.

mance measurement experiments in a heterogeneousenvironment: Pentium PCs, Sparc-5s, a 64-nodeMeiko CS-2 where the individual nodes are Sparc-10processors, and single and dual processor Ultra-Sparcs, connected by 10 and 100Mbit Ethernet. For amore detailed presentation of the performance resultsand the applications used the reader is referred to[14].

3.2.2. Raytracing measurementsWe have ported a sequential raytracer written in Java

to Javelin as an example of a real-world applicationthat benefits from additional processors even if com-munication is relatively slow. To evaluate the perfor-mance and dynamic behavior of our infrastructure wehave raytraced images of the size 1024× 1024 pixelsfor the two scenes shown in Fig. 3, and a randomlygenerated scene.

Fig. 4(a) shows the speedup curve for our par-allel raytracer running on simple and cone435 in acluster of two Sparc-5 s, five UltraSparcs, and onedual-processor UltraSparc. Fig. 4(b) gives the speedupcurve for raytracing random on the 64 Sparc-10 nodesof our Meiko CS-2. The graphs illustrate that thespeedup that can be achieved depends on the compu-tation to communication ratio. The more complex thescene, the longer it takes to compute the color of apixel, while communication costs stay the same andbecome less important overall.

3.2.3. Mersenne prime measurementsAs our second application we implemented a paral-

lel primality test which is used to search for Mersenne

prime numbers. This type of application is well suitedto Javelin, since it is very coarse-grained with a highcomputation-to-communication ratio when testinglarge Mersenne primes. For our measurements, wechose to test the Mersenne primality for all 119 primeexponents between 4000 and 5000. The reason forselecting this range is that on the one hand, we triedto make numbers large enough to simulate the trueworking conditions of the application, and on theother hand, we wanted to keep them small enoughto be able to complete our set of measurements in areasonable amount of time.

The first set of measurements was performed onclusters of identical machines. Fig. 5(b) presents thespeedup curve for test runs on a 64-node Meiko CS-2,while Fig. 5(a) shows the curve for a cluster of eightSun UltraSparcs. In both cases, the speedup was closeto linear as long as the ratio of job size to num-ber of processors was big enough. For large num-bers of processors communication through a singlerouter becomes a bottleneck. In a more realistic set-ting where huge primes are checked we do not ex-pect this to be a problem since the computation willbe extremely coarse-grained. For our tests, we chosea strategy where the biggest tasks (large amount ofcomputation) were enqueued at the beginning of thetask queue, thus ensuring that the hosts that partici-pate first get the largest amount of work. This led toan even task distribution.

In conclusion, the initial results from our pro-totype are highly encouraging. The measurementsshow that we can achieve almost linear speedup forsmall to medium numbers of hosts with this type ofcoarse-grained application. The next step is to seekways to avoid the saturation of the routing service andother bottlenecks in the system, so that the result willbe an infrastructure that truly scales to the size of theInternet. In the long term, it is our belief that the mostnatural way to avoid bottlenecks on a global scale isthe market-based resource allocation approach, sinceit has already proven itself superior to any centralizedscheme in the real world.

4. Current research

In this section, we present some possible approachesto tackle two of the most important issues in design-

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 665

Fig. 4. Speedup curves for raytraced images.

Fig. 5. Speedup curves for Mersenne primality test.

ing a global computing infrastructure,scalability andfault-tolerance. Without fault-tolerance, such a sys-tem is doomed to fail because of the very nature ofwide area networks with unstable communication andfrequent host failures. Without scalability, the systemloses its claim to the predicate ‘global’, thus defeatingits own purpose of bringing together large numbers ofhosts on the Internet.

Our goal in this phase of the project is to create a re-liable, stable infrastructure that scales to a larger num-ber of hosts than any comparable system has achievedso far.

We begin by giving an overview of work that hasbeen done in the field of fault-tolerance and thread mi-gration, followed by our proposal of a combination of

various techniques. Although the migration of threadsat runtime is not primarily linked to fault-tolerance,it is extremely useful in the setting of a so-called‘graceful retreat’, i.e., when a user suddenly reclaimsa host and the ongoing computation has to be savedin a portable state that can be restarted later on someother machine. As a side product, once the problem ofcheckpointing and migrating a thread has been solved,the problem of checkpointing for the sake of savingparts of a computation without actually migrating be-comes a subproblem of the more general case.

Later, we will talk about a possible way of en-hancing the scalability of the current Javelin proto-type based on a network of brokers that operate au-tonomously.

666 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

4.1. Fault-tolerance and thread migration

Traditional fault-tolerance techniques likecheck-pointing and transaction systemshave been studiedextensively in the database community. A good gen-eral reference is [15]. However, these mechanisms aregenerally perceived as costly and require the use of fileI/O (logging), which is often prohibitive in the globalcomputing setting.

One example of such a relatively costly approachis FT-Linda [16]. The original Linda definition ([17],see also Section 5) does not consider fault-tolerancemechanisms. FT-Linda is a version of Linda that ad-dresses these concerns by providing two kinds of en-hancements:stable tuple spaces, in which tuple valuesare guaranteed to persist across failures, andatomicexecution of tuple space operations, which allows forcollections of tuple space operations to be executed inan all-or-nothing fashion despite failures and concur-rent operations by other processes. To achieve thesefeatures, FT-Linda utilizes a replicated state machineapproach based on ordered atomic multicast. This willwork fine as long as the tuple space is replicated toonly a few machines in a LAN. Clearly, if the systemconsists of several thousand hosts, all with their ownlocal share of the storage space, atomic multicasts arenot a good solution. A similar concept is that of JavaS-paces [18], an extension to the Java language closeto Linda. Here, fault-tolerance is achieved through atransaction mechanism. Although this is generally avery useful approach to enhance the fault-tolerance ofJava itself, we believe that the following techniquesare more suitable in a global computing setting.

4.1.1. NotificationA simple, straightforward approach to a success-

ful fault-tolerance strategy is to leave the policy up tothe application and only provide a mechanism to de-tect faulty components. In many cases this might evenbe the only feasible solution for a system since theapplication semantics remain largely hidden from theinfrastructure. For instance, many traditional messagepassing applications have semantics that can only behandled by the application programmer, since only heor she knows how to handle lost messages or host fail-ures. The underlying system can only help by trying todetect such failures. A common tool for failure detec-

tion is anotification mechanismthat lets applicationsregister certaineventsthat are of interest to them andhave the system call a specified routine, a so-calledcallback function, when an event occurs. The callbackfunction can then handle the event according to theapplication semantics.

4.1.2. TIES/Eager schedulingTIES stands forTwo-Phase Idempotent Execution

Strategy. It is a fault-tolerance mechanism that wasfirst developed as a theoretical concept to transformarbitrary PRAM programs to execute on synchronousfail-stop PRAMs [19]. Later, the results were im-proved by combining probabilistic and deterministicalgorithms, and then extended to execution on asyn-chronous PRAMs where processors can be infinitelyslow. The basic idea of TIES is that faster processorsget rescheduled to subsume the work of slower pro-cessors, leading to correct execution since each stepis idempotent. In the asynchronous setting an addi-tional problem arises with respect to late writers, i.e.,processes that clobber memory locations by rewritingolder results. This can be overcome by either a centralinstance maintaining some notion of global time andrejecting late results or by not letting two processesthat work on the same parallel step write their resultsto the same location (memory evasion). The first prac-tical implementation of TIES was sketched in [20].The termeager schedulingwas coined around thattime, too, and is used as a synonym for TIES. Thestrategy is also used in the Charlotte/Knitting Factorysystem (see Section 5). The main advantage of eagerscheduling is its low overhead in the absence of fail-ures, and that it brings together parallel programming,load balancing, and fault-tolerance in a natural waythat fits the adaptively parallel programming model.

4.1.3. Portable checkpoints at byte code levelPortable Checkpointing [21] is a compiler based

technique that, by doing a source to source translationof ANSI-C code, instruments the source code to saveand recover from checkpoints that capture the stateof a computation in a machine independent format.This technique provides stack environment portabil-ity, enables the conversion of complex data types, andmakes pointers portable. The result is an efficient andcomfortable way of migrating threads between differ-

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 667

ent architectures. As a side product, traditional check-pointing without thread migration comes for free.

Compared to C, Java has several advantages with re-spect to this technique. Firstly, since Java has no point-ers, the whole problem of making pointers portableneed not be considered. Secondly, with Object Seri-alization already built in to the language, one doesnot have to deal with the task of storing data struc-tures in a portable manner. Java’s multithreaded con-cept also seems more appropriate from the begin-ning. All this leads us to believe that the techniquecan be extended to Java in a fairly straightforwardmanner.

In order to hide this code conversion from the ap-plication programmer we are currently investigatingPortable Checkpointing at bytecode level. If this issuccessful it would open up the possibility of makingthe conversion at the broker instead of the client’smachine, and the bytecode could be shipped as amethod parameter in an RMI call. The broker couldthen make selective decisions on which hosts shouldreceive unmodified bytecode and which hosts shoulduse a checkpoint version due to their instability.

4.1.4. Unified approachAfter studying the various techniques for fault-

tolerance and their costs and benefits in a globalcomputing setting, we have come to the followingconclusions.

Any broadcast/multicast based technique involvingreplication is too costly due to network latency andbandwidth constraints and the potentially unlimitednumber of hosts. Logging based techniques are alsoruled out due to high overhead and lack of local fileI/O.

Checkpointing is a good way to preserve partialwork results and restart applications after failures.However, it normally requires local file I/O whichcollides with the Java security model. Otherwise, ifcheckpointing involves communication, it may be toocostly depending on the nature of the application. Inthe case of long running applications like MersennePrimes, checkpointing might be essential becauserestarting jobs after failures would be unacceptable.All this leads us to believe that a good approach wouldoffer checkpointing as a tool for the application toselect and configure.

A notification mechanism that leaves the fault-tolerance policy up to the application is consideredhighly beneficial. Having a separate thread moni-tor the status of a host and take action based on awell-defined event model is also beneficial to the sys-tem itself, e.g., in case of broker failures or networkproblems.

Eager scheduling is a relatively simple and effi-cient way to ensure fault-tolerance. However, it mightbe costly with long running applications in the pres-ence of frequent failures. Consider once again theMersenne Prime example: If failures are frequent,jobs might run for several hours, then fail. If, in themean time, the same job were scheduled over andover again without knowing whether the first host willcomplete it successfully, the system would be wastinga lot of valuable resources. In this case, checkpoint-ing seems the superior method. In fact, checkpointingand eager scheduling seem to be on opposite endsof the same scale – the longer the individual job,the more valuable checkpointing becomes, and viceversa.

Portable Checkpointing is highly desirable since itopens up the possibility of a true ‘graceful retreat’, i.e.,if a user reclaims a host and the system needs to vacateall resources quickly, it can pack up the computationthread and move its state to the broker. From there, itcan be rescheduled to the next idle host and restartedexactly where it left off.

As a consequence, we suggest a combined approachto the fault-tolerance problem that will lead to aToolkitAPI for the application programmer. The unified strat-egy will consist of three parts:1. An Event Managerthread that runs on every

Javelin host and monitors events that applica-tions and the system are interested in. If an eventis triggered, an asynchronous message is sentto the interested party and an appropriatecall-back methodis invoked to handle the event. Anexample of such an event model, although some-what elaborate and designed for a very differentsetting, is given in [22].

2. Eager Schedulingwill serve as the default mech-anism if nothing else is specified by the applica-tion.

3. The application will be able to selectPortableCheckpointingand configure the regular check-pointing interval according to its needs.

668 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

Once the implementation is ready, further experi-ments are needed to evaluate the different techniquesand their suitability for specific applications.

4.2. Scalability

To be clear about what we mean byscalable, ifa global computational infrastructure is scalable, itscomponents have bounded power: bounded computa-tional rate, bounded communication rate, and boundedstate. In particular, for Javelin to be scalable, its clients,brokers, and hosts have bounded power. These boundsimply that, for example, clients, brokers, and hosts,can communicate with only a fixed number of othercomponents during a fixed interval of time. Thus, atany point in time, there are bounds on the number ofconnections between hosts, between brokers, betweenbrokers and hosts, and between the client and brokers.Bounded state similarly implies bounds on the num-ber of brokers that a broker can know about at anypoint in time.

The prototype offers just a single broker/router thatbecomes a bottleneck when too many hosts partic-ipate in a computation. Clearly, a network of bro-kers must be realized in order to achieve scalability.Internet-wide participation means that all hosts mustbe largelyautonomousand able to work in the pres-ence of node and network failures. Scalability impliesthat the architecture cannot be centralized. Boundedstate implies that no site can, in general, have a globalsystem view (e.g., a table with the names of all partic-ipating brokers). We have identified two key problemsin building a scalable architecture:1. Host allocation and code distribution – How does a

client find hosts for its computation, and how doesthe code get distributed efficiently to a potentiallyvery large number of hosts?

2. Data communication at runtime – How is data ex-changed between participating hosts after an ap-plication has been successfully started?

In the following we describe how these problemsmight be solved. Technically, the proposed systemwill be implemented entirely in Java. As the underly-ing distributed object technology, Java RMI (RemoteMethod Invocation) will be used. The following dis-cussion is based on a scenario where all hosts executecode as Java applets. Therefore, the system must deal

Fig. 6. State transition diagram for Javelin hosts.

with applet security restrictions e.g., no local file I/O.We will, however, assume that applets can communi-cate directly with one another once their RMI handleshave been passed along through the server2 .

The rest of this section is structured accordingto the different states a Javelin host can be in dur-ing its lifetime. The complete state transition di-agram is shown in Fig. 6. There are four states:NoHost, Standby, Ready , and Running . If ahost has not joined Javelin it is in stateNoHost .The transition toStandby is made by contactinga broker, registering, and downloading the daemonthread. In the next section we will describe how hostscan be allocated and code can be shipped so that anapplication is ready to start up, causing a state tran-sition from Standby to Ready . In Section 4.2.2we will present the data exchange mechanism basedon a distributed, double ended queueand addresshashing, which allows the host to run the applicationand therefore transition toRunning . The diagramhas two more sets of transitions, a ‘natural’ way backfrom each state to the previous state when a phase hasterminated, and a set of ‘interrupt’ transitions (shownin dashed lines) that lead back to theNoHost statewhen a user withdraws the host from the system.

4.2.1. Finding hosts and shipping codeAs stated above, a client and its local broker do not

actively look for hosts to join a computation. Hosts canjoin at any time, either by contacting the same brokeras the client or indirectly through some other broker.Hosts are managed by brokers according to thefarm-ing principle – each broker can only accommodate acertain number of hosts, depending on its performance

2 This method has been proposed and successfully tested in theKnitting Factory project [23] and runs with Sun’s JDK 1.1 and theHotJava browser. Since at this point the two leading web browsers,Netscape Communicator and Microsoft Internet Explorer, bothimplement their own security policies and subsets of JDK 1.1,there currently is no agreed standard security model.

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 669

characteristics. Brokers can communicate with otherbrokers, but they will only be able to hold a limitednumber of broker addresses in their working set atany time since the number of participants is poten-tially very large and the address table is bounded insize.

If every host that participates in a computation hadto go to the client to download the code this wouldsoon lead to a bottleneck for large numbers of hosts.Therefore, first the local broker and then every otherbroker that joins in a computation will act as acacheon behalf of the client. Thus, client code has to beuploaded first to the local broker and then passed onto every other broker requesting work on behalf of itsidle hosts. The uploading can be realized by recurs-ing through the class call graph and packing all nec-essary classes into a single archive file. Upon termi-nation of a computation, the host that detects termina-tion (usually the original client) must send an invali-dation message to its broker, which in turn passes it onto all other brokers involved, so that the code can bedeleted.

Technically, each broker will be implemented as aJava RMI server, so that it can directly call methods onanother broker object. Java Object Serialization willbe helpful in the actual shipping of the code. Hostswill run as applets, but will also be RMI servers sothey can be contacted by other hosts.

In the following we describe the sequence of stepsfrom the moment a client application is willing to ex-ecute until the moment when a host has received thecode to participate in the computation.1. The client contacts its local broker.2. If the broker is willing to accept jobs, the client

wraps up all necessary code and uploads it to thelocal broker. Depending on the type of application,the client may now start up and execute on its own.

3. The local broker puts the code to disk and createsan HTML page so that the code can be down-loaded as an applet. In doing so, the broker actsas adisk cachefor the client.

4. A host joins the system by pointing its browser toits local broker’s URL. In doing so, it downloadsa daemon applet from the broker that will man-age host computation and communicate with thebroker.

5. The host daemon contacts the local broker askingfor code to execute.

6. (a) If the local broker has work, it sends a codeURL back to the host. If not, it contacts anotherbroker and asks for code. To preserve the prin-ciple of locality and avoid flooding the networkwith concurrent requests from a single broker, wesuggest usingbreadth-first searchas the search al-gorithm. (b) If the other broker has code, it sendsback the code in reply. The local broker saves thecode to disk and creates an HTML wrapper as be-fore. The local broker can now send the new URLback to the host daemon.

7. The host daemon executesshowURL() to forcethe browser to load the code applet.

8. The application starts to execute on this host.We believe that this method will meet the system

goals. To back our claim, consider the following:Scalability is potentially unlimited, since each bro-

ker is responsible only for a bounded number of hostsand can communicate with a bounded set of other bro-kers. Locality is maximized, because the system willalways find hosts in the vicinity of the client first byusing BFS (ideally, the local hosts will be sufficient,but next it will find hosts that are one broker hop away,then two hops, and so on). Ease of use remains opti-mal for participating hosts – the mechanism of retriev-ing code from other brokers is completely transpar-ent. Autonomy is guaranteed, since each broker canmake its own decisions on when and how often to callother brokers for jobs. The chosen network topology,an unrestricted graph of bounded degree, can resultin a high degree of network connectivity if the brokerconnections are chosen carefully to prevent articula-tion points. This compares favorably to any strictlytree-based or hierarchical scheme, like the one usedin the ATLAS project (see Section 5). This allows theJavelin network to remain functional even in the pres-ence of failures, greatly enhancing its fault-tolerancecapabilities.

4.2.2. Distributing and exchanging dataAs stated before, Javelin supports two differ-

ent programming models: Fixed Parallelism withmessage passing, and Adaptive Parallelism withmaster-worker-subcontracting. The following discus-sion is restricted to the latter case, since the messagepassing model is more geared towards letting theapplication choose all its data structures and commu-

670 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

nication patterns. The system really cannot make anydecisions since the application semantics are hiddenfrom it at that level.

In the subcontracting model we base our strategy ontwo main data structures that are local to every host:a hash tableof host addresses (technically, Java RMI-handles), and adistributed, double-ended task queuecontaining ‘chunks of work’. From the point of view ofscalability, using a hash table allows for fast retrievalin the average case and scales to very large numbers.In addition, there is no centralized site in this setup,and host autonomy is guaranteed since sufficient in-formation is kept locally to remain functional in thepresence of failures. It is important to observe that theaddress table isbounded in size– the hash table is pre-allocated to some fixed size that remains manageable.At startup, a broker’s connections are preconfiguredto some default setup, whereas a host’s first connec-tion is always its local broker. Over the lifetime of thehost, the address table will fill up through interactionwith other hosts. Should the table eventually overflow,the need to evict ‘older’ connections will arise, whichcan be taken care of by a standard replacement pol-icy like LRU. All this will result in a working setofconnections for each host.

The task queue is double-ended because we fol-low the concept ofrandomized work stealingwhichwas first introduced in the Cilk project ([24], see alsoSection 5). The local host picks work off one end ofthe queue, whereas remote requests get served at theother end. Jobs get split until a certain minimum gran-ularity determined by the application is reached, thenthey will be processed. This means that when a hostruns out of local jobs, it picks one of its neighbors atrandom from its hash table and issues a work requestto that host. In doing so the host piggybacks its ownaddress information onto the request and selects thenumber of hops that the request shall be forwarded ifunsuccessful. This is to make sure unsuccessful workrequests actually die eventually and will not floodthe network. Fig. 7 shows the principle of the taskqueue.

5. Related work

In this section we summarize the research effortsthat are most closely related to our work. At present,

Fig. 7. Distributed double-ended task queue.

none of these projects have shown that they will scaleto very large numbers of participants. In fact, justlike the current Javelin prototype, most of these de-signs are hampered by some centralized system com-ponent that becomes a bottleneck for large numbers ofhosts.

Charlotte/Knitting Factory [23,25] supports dis-tributed shared memory, and uses a fork-join modelfor parallel programming. A distinctive feature ofthis project is itseager schedulingof tasks, where atask may be submitted to several servers, providingfault-tolerance and ensuring timely execution. Char-lotte provides fault-tolerance based on the fact thateach task is atomic. Changes to the shared memorybecome visible only after a task completes suc-cessfully. This allows a task to be resubmitted to adifferent server, in case the original server fails.

ATLAS [26] provides a global computing modelbased on Java and on the Cilk programming model[24] that is best suited to tree-like computations.Cilk is a C-based runtime system for multithreadedparallel programming on traditional MPPs, althoughvariants have been implemented in other languagesand on more loosely coupled workstation clusters(see ATLAS above). It provides a distinct program-ming model, where a Cilk program consists of acollection of procedures, each of which is brokeninto a sequence of nonblocking threads. Since threadsare nonblocking, they can always run to completiononce invoked. Threads can spawn either children orsuccessors, with the latter needed to pick up data

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 671

dependencies from any children of a predecessor.The programming model is well suited to asyn-chronous, tree-like computations. Cilk also introducesa work-stealing scheduler that enables a processorthat runs out of work to steal work from another busyprocessor, leading to efficient load balancing in thesystem. ATLAS ensures scalability using a hierarchyof managers. The current implementation uses nativelibraries, which may raise some portability problems.Like Charlotte, ATLAS provides fault-tolerance basedon the fact that each task is atomic. Each subtask iscomputed by a subtree in the hierarchy of servers.Any subtask that does not complete times out andis recomputed from a checkpoint file local to itssubtree.

Popcorn [27] provides a Java API for writingparallel programs for Internet distribution. Applica-tions are decomposed by the programmer into small,self-contained subcomputations, calledcomputelets.The application does not specify a destination onwhich the computelet is to execute. Rather, a cen-tralized entity called ‘market’ brings together buyersand sellers of CPU and determines which seller willrun the computelet. Market incentives are supported,e.g. two different types of auction for CPU cycles.User participation as a seller is made extremely easy– just point a Java-enabled browser to the marketweb site and fill in a user profile to open an accountfor so-called ‘Popcoins’, the micro-currency used byPopcorn.

Bayanihan[28] identifies issues and concepts of aJava-based system like Javelin. It classifies the partici-pation in a global computing framework into differentlevels ofvolunteer computing, touching on economicconcepts (e.g. barter for CPU time without proposinga broker model). The current prototype provides a gen-eral framework for executing tasks within a so-called‘chassis object’ that can either be a Java applet or ap-plication. Tasks are dealt out by a centralized server(work manager) and executed by work engines in theclient. Communication in Bayanihan is based on theHORB [29] distributed object library. Scheduling andfault-tolerance schemes are left to the application pro-grammer.

Ninflet [30] is a Java-based global computing sys-tem with an architecture very similar to Javelin.Clientscorrespond to Javelin clients,Dispatcherscor-respond to Javelin brokers, andServerscorrespond to

Javelin hosts. One major difference is that the sys-tem is based on Java applications instead of applets,thus overcoming applet security restrictions and let-ting the computational units, calledNinflets, establishpoint-to-point RMI communication. A special Ninfletsecurity model is provided to address security con-cerns that result from the use of applications. Theprogramming model is a master-slave model in whichthe client acts as the master. A special feature of theNinflet system is the ability to checkpoint ongoingcomputations in order to provide fault-tolerance aswell as graceful evacuation of hosts reclaimed by theirusers.

Historically, the forerunners of the current groupof global or metacomputing systems have beenLAN-based parallel computing packages using themessage passing paradigm, like e.g. PVM [7] or MPI[31]. Recently, some research projects have built uponthis approach by keeping the programming modeland enhancing interoperability through languageslike Java and object orientation. Successful examplesare JPVM [32], IceT [33], and ParaWeb [34]. Thefollowing systems require the user to have login ac-cess to all participating machines and binaries for allarchitectures used in the computation.

Piranha [35] is one of the first projects aiming atutilizing idle computers in a LAN for parallel appli-cations. It introduces adaptive parallelism, in whichthe number of processes participating in a computa-tion may vary at runtime. The programming modelis a master-worker approach; task communication ishandled by a Linda tuple space [17]. The system isfault-tolerant by reassigning tasks that have not beencompleted due to host withdrawal. Linda is a languagefor programming parallel applications whose most no-table feature is a distributed shared memory called tu-ple space. A collection of primitives operating on thistuple space allow for interprocess communication andsynchronization. Linda implementations are availableon a number of different architectures and for a num-ber of different languages.

Globus[36] is viewed as a networked virtual super-computer also known as a metacomputer: an executionenvironment in which high-speed networks are usedto connect supercomputers, databases, scientific in-struments, and advanced display devices. The projectaims to build a substrate of low-level services – such ascommunication, resource location and scheduling,

672 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

authentication, and data access – on which higher-levelmetacomputing software can be built.

Like Globus,Legion [37] strives to provide a sin-gle, coherent virtual machine that can accommodatelarge numbers of hosts. Some of the design goals arescalability, programming ease, fault-tolerance, site au-tonomy, and multi-language support. To achieve thesegoals and a single, persistent name space, Legion intro-duces an object model that wraps around every com-ponent of the system.

6. Conclusion

We have designed and implemented a prototypeof Javelin, an infrastructure for Internet-based par-allel computing using Java. Javelin allows machinesconnected to the Internet to make a portion of theiridle resources available to remote clients, and at othertimes use resources from other machines when morecomputational power is needed. In order to be suc-cessful, an infrastructure for transforming the Webinto an immense parallel computing resource must beeasy to install and use. By requiring clients and hoststo have access to only a Java-enabled Web browser,Javelin achieves ease of participation: No OS or com-piler modifications are needed and system administra-tive costs are zero. We presented experimental resultsfor two coarse-grain, compute-intensive applications:a raytracer and a Mersenne primality test. For both ap-plications we achieved good speedups on Javelin. Weexpect that future versions of Javelin will see a per-formance boost due to optimized JIT compilers. Weoutlined pressing research issues in fault-tolerance andscalability, properties that are essential to any globalcomputing infrastructure.

Overall, we believe that Javelin can be used suc-cessfully for Internet-based parallel computation, andfuture versions will address the remaining challengesto establishing a robust global computing infrastruc-ture.

References

[1] I. RSA Data Security, The RSA Data Security Secret-KeyChallenge, 1997. http://www.rsa.com/rsalabs/97challenge.

[2] DESCHALL, Internet-Linked Computers Challenge DataEncryption Standard, Press Release, 1997. http://www.frii.com/ rcv/despr4.htm.

[3] Colusa Software, Omniware Technical Overview, 1995.http://www.colusa.com.

[4] I. Goldberg, D. Wagner, R. Thomas, E.A. Brewer, A SecureEnvironment forUntrusted Helper Applications – Confiningthe Wily Hacker, Proceedings of the 1996 USENIX SecuritySymposium, 1996.

[5] Electric Communities, The E Programming Language, 1996.http://www.communities.com/e/epl.html.

[6] Lucent Technologies Inc. Inferno. http://inferno.bell-labs.com/inferno/.

[7] V.S. Sunderam, PVM: A Framework for Parallel DistributedComputing, Technical Report ORNL/TM-11375, Departmentof Mathematics and Computer Science, Emory University,Atlanta, GA, USA, February, 1990.

[8] T. Wilkinson, Kaffe – A free virtual machine to run Java code,1997. http://www.tjwassoc.demon.co.uk/kaffe/kaffe.htm.

[9] Softway, Guava – Softway’s just-in-time compiler for Sun’sJava language. http://guava.softway.com.au/.

[10] R. Wolski, N. Spring, C. Peterson, Implementing aPerformance Forecasting System for Metacomputing: TheNetwork Weather Service, Proceedings of the ACM/IEEEConference on Supercomputing (SC97), San Jose, CA,November 1997.

[11] J. Feigenbaum, Encrypting Problem Instances – Or,. . . , CanYou Take Advantage of Someone Without Having to TrustHim? Proceedings of the CRYPTO’85 Conference, 1985.

[12] A. Alexandrov, M. Ibel, K.E. Schauser, C. Scheiman,SuperWeb: research issues in Java-based global computing,Concurrency: Practice and Experience 9(6) (1997) 535–553.

[13] R. Saavedra-Barrera, A. Smith, E. Miya, Machinecharacterization based on an abstract high-level language,IEEE Trans. Comput. 38(12) (1989) 535.

[14] B.O. Christiansen, P. Cappello, M.F. Ionescu, M.O. Neary,K.E. Schauser, D.Wu. Javelin, Internet-based parallelcomputing using Java, Concurrency: Practice and Experience9(11) (1997) 1139–1160.

[15] P.A. Bernstein, V. Hadzilacos, N. Goodman, ConcurrencyControl and Recovery in Database Systems, Addison-Wesley,1987.

[16] D.E. Bakken, R.D. Schlichting, Supporting fault-tolerantparallel programming in Linda, IEEE Trans. ParallelDistributed Systems 6(3) (1995) 287–302.

[17] R.A. Whiteside, J.S. Leichter, Using Linda forSupercomputing on a Local Area Network, Technical ReportYALEU/DCS/TR-638, Department of Computer Science, YaleUniversity, NewHaven, CT, 1988.

[18] Sun Microsystems, Inc. JavaSpace Specification, March,1998. Revision 1.0.

[19] Z. Kedem, K. Palem, P. Spirakis, Efficient robust parallelcomputations, Proceedings of the 22nd ACM Symposium onTheory of Computing, 1990.

[20] P. Dasgupta, Z. Kedem, M. Rabin, Parallel processing onnetworks of workstations: A fault-tolerant high performanceapproach, Proceedings of the 15th IEEE InternationalConference on Distributed Computing Systems, 1995.

M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674 673

[21] B. Ramkumar, V. Strumpen, Portable Checkpointing forHeterogeneous Architectures, 27th International Symposiumon Fault-Tolerant Computing Digest of Papers, 1997, pp.58–67.

[22] M.O. Neary, D. Schumacher, DOMS – A Prototype of aDistributed, Object Oriented, Active Database Kernel as aFramework for Cooperative Transactions. Master’s thesis,Universität-Gesamthochschule Paderborn, Germany, 1995.

[23] A. Baratloo, M. Karaul, H. Karl, Z.M. Kedem, AnInfrastructure for Network Computing with Java Applets,Proceedings of the ACM 1998 Workshop on Java forHigh-Performance Network Computing, Palo Alto, CA,February, 1998.

[24] R.D. Blumofe, C.F. Joerg, B.C. Kuszmaul, C.E. Leiserson,K.H. Randall, Y. Zhou, Cilk: An Efficient MultithreadedRuntime System, 5th ACM SIGPLAN Symposium onPrinciples and Practice of Parallel Programming (PPOPP ’95)207-216, Santa Barbara, CA, July, 1995.

[25] A. Baratloo, M. Karaul, Z. Kedem, P. Wyckoff, Charlotte:Metacomputing on the Web, Proceedings of the 9thConference on Parallel and Distributed Computing Systems,1996.

[26] J.E. Baldeschwieler, R.D. Blumofe, E.A. Brewer, ATLAS:An Infrastructure for Global Computing, Proceedings ofthe Seventh ACM SIGOPS European Workshop on SystemSupport for Worldwide Applications, 1996.

[27] N. Camiel, S. London, N. Nisan, O. Regev, The POPCORNProject: Distributed Computation over the Internet in Java,6th International World Wide Web Conference, April,1997.

[28] L.F.G. Sarmenta, Bayanihan: Web-Based Volunteer Com-puting Using Java, 2nd International Conference onWorld-Wide Computing and its Applications, Mar. 1998.

[29] S. Hirano, HORB: Extended Execution of Java Programs,First International Conference on World-Wide Computingand its Applications (WWCA 97), 1997.http://ring.etl.go.jp/openlab/horb/.

[30] H. Takagi, S. Matsuoka, H. Nakada, S. Sekiguchi,M. Satoh, U. Nagashima, Ninflet: a Migratable ParallelObjects Framework using Java, Proceedings of the ACM1998 Workshop on Java for High-Performance NetworkComputing, Palo Alto, CA, Feb. 1998.

[31] M. Snir, S.W. Otto, S. Huss-Lederman, D.W. Walker, J.Dongarra. MPI, The Complete Reference, MIT Press, Nov.1995.

[32] A.J. Ferrari. JPVM: Network Parallel Computing in Java,Proceedings of the ACM 1998 Workshop on Java forHigh-Performance Network Computing, Palo Alto, CA,1998.

[33] P.A. Gray, V.S. Sunderam, Native-Language-Based Dis-tributed Computing Across Network and FilesystemBound-Aries, Proceedings of the ACM 1998 Workshop onJava for High-Performance Network Computing, Palo Alto,CA, 1998.

[34] T. Brecht, H. Sandhu, M. Shan, J. Talbot, ParaWeb: TowardsWorld-Wide Supercomputing. Proceedings of the SeventhACM SIGOPS European Workshop on System Support forWorldwide Applications, 1996.

[35] N. Carriero, D. Gelernter, D. Kaminsky, J. Westbrook,Adaptive Parallelism with Piranha, Technical ReportYALEU/DCS/TR-954, Department of Computer Science, YaleUniversity, NewHaven, Connecticut, 1993.

[36] I. Foster, C. Kesselman, Globus: A MetacomputingInfrastructure Toolkit, Int. J. Supercomputer Appl. 1997.

[37] A.S. Grimshaw, W.A. Wulf, The Legion team, The LegionVision of a Worldwide Virtual Computer, Communicationsof the ACM, 40 (1) (1997).

ProfessorCappello received the Ph.D de-gree in Computer Science from PrincetonUniversity in 1982. He is currently inves-tigating global computing, and market re-source allocation mechanisms. He is alsointerested in self-directed learning. He haspublished journal articles on integer multi-plication, digital signal processing, parallelalgorithms, systolic algorithms, and opti-mal multiprocessor schedules. Drs. Cap-

pello and Miranker received a US and a European patent on be-half of the NSF, the University of California, and IBM for in-ventions related to a floating-point device. He is a senior memberof the IEEE, and a member of the ACM, and of the AdvisoryBoard of the IEEE Signal Processing Society’s TC on Designand Implementation of Signal Processing Systems, of which heserved as founding Chair. He chaired in 1984 IEEE Workshopon VLSI Signal Processing, and co-edited VLSI Signal Process-ing. He was Program Co-Chair for the 1994 Int. Conf. On Ap-plication Specific Array Processors (ASAP’94), and ConferenceCo-Chair for ASAP’95. Dr. Cappello is a past Associate Editorof the IEEE Tran. On Acoustics, Speech, and Signal Process-ing, and is on the editorial board of the Journal of VLSI SignalProcessing.

Bernd Oliver Christiansen is a PhDcandidate in the Department of Com-puter Science at the University of Cali-fornaia at Santa Barbara. He received hisDiploma in Computer Science with dis-tinction from the University of Hamburg,Germany in 1996. Bernd’s research inter-ests are in Market-based control of dis-tributed systems, wireless communication,and data compression. Bernd has received

a highly competitive scholarship of the Deutscher Akademis-cher Austauschdienst (German Academic Exchange Service) andthe Mericus fellowship. He helped with the organization of theACM 1998 Workshop on Java for High Performance NetworkComputing.

674 M.O. Neary et al. / Future Generation Computer Systems 15 (1999) 659–674

Michael Oliver Neary is currently a PhDstudent in the Department of ComputerScience at the University of California atSanta Barbara. He received the Diploma inComputer Science from the University ofPaderborn, Germany, in March 1995. Hiscurrent research is focusing on Java-basedglobal computing, with special emphasison fault-tolerance and scalability. Other in-terests include parallel and distributed sys-

tems, in particular distributed database systems and the issues ofevent management and cooperative transactions.

Dr. Klaus Schauseris an Associate Pro-fessor in the Department of Computer Sci-ence at the University of California atSanta Barbara. He received his diploma inComputer Science with distinction fromthe University of Karlsruhe in 1989 and theM.S. and Ph.D. degrees, also in ComputerScience, from the University of California,Berkeley, in 1991 and 1994, respectively.Professor Schauser’s research interests are

in efficient communication, cluster computing, parallel languages,systems for highly parallel architectures, and Web-based globalcomputing. He is the recipient of the NSF CARRER Award, theUC Regent’s Junior Faculty Fellowship, the Distinguished Lec-tureship from the German American Academic Council, and var-ious teaching awards. Dr. Schauser has served on many programcommittees and was the program co-chair for the ACM 1998Workshop on Java for High Performance Network Computing.