CompuP2P: An Architecture for Internet Computing Using Peer-to-Peer Networks

CompuP2P: An Architecture for InternetComputing Using Peer-to-Peer Networks

Rohit Gupta, Varun Sekhri, and Arun K. Somani, Fellow, IEEE

Abstract—Internet computing is emerging as an important new distributed computing paradigm in which resource intensive computing

is integrated over Internet-scale networks. Over these large networks, different users and organizations share their computing

resources, and computations take place in a distributed fashion. In such an environment, a framework is needed in which the resource

providers are given incentives to share their resources. CompuP2P is a lightweight architecture for enabling Internet computing. It uses

peer-to-peer networks for sharing of computing resources. CompuP2P create dynamic markets of network accessible computing

resources, such as processing power, memory storage, disk space, etc., in a completely distributed, scalable, and fault-tolerant

manner. This paper discusses the system architecture, functionality, and applications of the proposed CompuP2P architecture. We

have implemented a Java-based prototype, and our results show that the system is light-weight and can provide almost a perfect

speedup for applications that contain several independent compute-intensive tasks.

Index Terms—Peer-to-peer, distributed computing, selfishness, game theory, microeconomics.

Ç

1 INTRODUCTION

INTERNET computing is a distributed computing paradigmthat uses the Internet as a single large virtual computer.

Internet computing promises to fulfill the vision of “any-time” “anywhere” computing. Applications benefiting fromthis paradigm range from simple data sharing to ones usingthe Internet as a processing engine for large-scale datastorage and distributed task execution. Internet computingis challenging to realize primarily because of its sheer sizeand open untrusted environment. In the last decade, theconcept of Internet computing has been revolutionized dueto applications such as file sharing developed around thepeer-to-peer (P2P) paradigm. We believe that the P2Pparadigm has the potential to serve as a platform fordeveloping several “killer-apps” for making true Internetcomputing a reality (and also affordable). Peer-to-peer(P2P) networks [1], [2], [3], [5], [6] are flexible distributedsystems that allow nodes (also called peers) to act as bothclients and servers and provide services to each other. P2Pis a powerful emerging networking paradigm that permitsthe sharing of virtually unlimited data and computationalresources in a completely distributed, fault-tolerant, scal-able, and flexible manner.

While the Internet currently cannot hope to serve as atotally general-purpose efficient parallel computer, it canstill serve as an excellent platform with unlimited computa-tional resources for solving a wide variety of computationalproblems. Some interesting applications for CompuP2P aregiven as follows:

1. Allowing processing-limited devices, such as wire-less clients, to distribute their processing require-ments to other machines in the network.

2. Utilizing the storage capacity of virtually millions ofmachines connected to the Internet, etc.

The prospect of harnessing a significant fraction ofnetwork resources to execute a distributed application isvery appealing. These resources can be purchased ondemand, in very short periods of time, and for very short(and larger when necessary) periods of time. Resources willgenerally be owned by different users that will notnecessarily volunteer to make them freely available, evenwhen they are not being used. To provide the incentives forlarge-scale resource sharing, resources need to be buyableand sellable, with the possibility of contracts that (at least intheory) can be enforced.

To enable large-scale resource sharing, a market econo-my-based framework is needed so that computing re-sources can be traded on demand in short periods of time.This would give incentive to individuals or large organiza-tions to share their compute resources. This paper discussesthe system architecture, functionality, and applications ofthe proposed CompuP2P architecture. CompuP2P usespeer-to-peer networks for sharing of computing resources.It creates dynamic markets of network accessible computingresources, such as processing power, memory storage, diskspace, etc., in a completely distributed, scalable, and fault-tolerant manner. CompuP2P use ideas from game theory [9]and microeconomics [12] to devise incentive-based schemesfor motivating users to share their computing resourceswith each other.

CompuP2P is designed to be lightweight in terms ofprotocol overhead and simplicity in control mechanisms. Ituses P2P substrate, a Chord [5] overlay network, to providelower level services, for example, connecting peers. Abovethis substrate, different lightweight control mechanismshave been built to create markets dynamically. A search for

1306 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 17, NO. 11, NOVEMBER 2006

. R. Gupta and A.K. Somani are with the Dependable Computing andNetworking Laboratory, Department of Electrical and Computer Engineer-ing, Iowa State University, Ames, IA 50011.E-mail: {rohit, arun}@iastate.edu.

. V. Sekhri is with Microsoft, One Microsoft Way, Redmond, WA 98052-6399. E-mail: [email protected].

Manuscript received 10 Jan. 2005; revised 24 June 2005; accepted 27 Dec.2005; published online 26 Sept. 2006.Recommended for acceptance by J. Wu.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number TPDS-0010-0105.

1045-9219/06/$20.00 � 2006 IEEE Published by the IEEE Computer Society

appropriate markets, trading, and pricing of resources isdone in a completely distributed manner without requiringany trusted and/or centralized authority to oversee thetransactions. The number of messages exchanged to look upan appropriate market is OðlogNÞ, where N is the numberof nodes in a network.

We have implemented a Java-based prototype ofCompuP2P architecture, and our results show that thesystem is lightweight, and can provide almost a perfectspeedup for applications that contain several independentcompute intensive tasks. For concreteness, in this paper, weuse compute power as the resource under consideration,however, the mechanism for market creation and resourcepricing is equally applicable to any other kind of resource,such as disk space, etc.

The rest of the paper is organized as follows: Section 2compares Internet computing with grid computing andpublic resource computing. Section 3 gives an overview ofCompuP2P system architecture. Section 4 describes theprotocols for market creation and resource pricing. InSection 5, we present our prototype implementation ofCompuP2P, while in Section 6, we draw differencesbetween CompuP2P and other existing large, scale dis-tributed computing projects. We conclude the paper inSection 7.

2 GRID COMPUTING AND PUBLIC RESOURCE

COMPUTING

Internet computing along with grid computing and publicresource computing share the goal of better utilizingexisting computing resources. However, there are profounddifferences among the three paradigms, and it is unlikelythat current grid middleware or public resource sharingarchitectures [7] will be suitable for Internet computing.

Grid computing [8] involves organizationally-ownedresources: supercomputers, clusters, and PCs owned byuniversities, research labs, and companies. These resourcesare centrally managed by IT professionals, are powered onmost of the time, and are connected by high bandwidthnetwork links. Malicious behavior, such as intentionalfalsification of results, are handled outside the system,e.g., by using a legal system.

Public resource computing [7] involves an asymmetricrelationship between projects and participants. Projects aretypically small academic research groups with limitedcomputer resources, expertise, and manpower. Most parti-cipants are general Internet users with PCs, workstations,etc., with low bandwidth connectivity to the Internet. Thecomputers are frequently turned off or disconnected fromthe Internet. Participants contribute their resources eitherout of altruism or they receive suitable “credit” for doing so.Projects have no control over participants, and cannotprevent malicious behavior.

In contrast, the Internet computing paradigm aims tocreate a single large heterogeneous pool of computingresources into which users can tap into to carry out theirtasks. Here, users can include enterprises, research groups,or even individual home PC owners. The system is typicallylarge with thousands or even millions of users. Network

connectivity, as in public resource computing, is sporadic.There is no centralized entity that controls the behavior ofindividual users and, thus, users can be expected to behaveselfishly (and even maliciously). Due to the large-scale,dynamism, openness, and heterogeneity of these systems,building a platform for Internet computing present severalunique and interesting research challenges. These issues,along with how they are addressed by the CompuP2Parchitecture, are discussed throughout the remainder of thispaper.

3 COMPUP2P: SYSTEM OVERVIEW

CompuP2P uses a peer-to-peer architecture for creatingmarkets for trading of computing resources such as CPUcycles, disk space, etc. CompuP2P create different marketsfor different amounts of a computing resource, referred toas a commodity. Nodes that are responsible for runningdifferent commodity markets are termed as “market own-ers” (MOs). MOs are dynamically reassigned as nodesleave and join the network. A MO does the job of amatchmaker between sellers and buyers, and maintaininformation about the sellers. This information can includethings such as available compute power and/or disk space,operating system type and version, platform type, pricerequirement, etc. Upon receiving a request from a client, theMO returns the information about the seller that best meetsthe client’s requirements. A Chord-based protocol is usedfor the market’s creation and lookup, and with highprobability, both sellers and buyers of a commodityconverge on the same market, i.e., both sellers and buyerscontact the same MO that is responsible for running themarket for that commodity. It must be noted that a singlephysical node can be a MO for various commodities.

CompuP2P is designed to take into account users’selfishness, and use ideas from game theory and micro-economics for pricing of computing resources. CompuP2Pallow users to define their policies regarding what, when,how, and by whom their resources can be used. Moreover,it allow users to specify their task requirements whileaccessing the system resources. For example, a user canspecify the timeliness and reliability requirements regard-ing the received results.

Fig. 1 depict the layers constituting the CompuP2Parchitecture. The functionality of these layers are explainedin the following sections. Fig. 1 depicts a conceptual view ofthe operation of a CompuP2P system.

3.1 Computing Resource Layer

This layer refers to various distributed resources, such ascompute power, disk space, files, etc., that exist in any largeInternet-scale system. These resources belong to differentnodes that are part of the underlying P2P network. In ourprototype implementation of CompuP2P, nodes joining thesystem are organized in a Chord ring [5]. Although, wehave used Chord as the underlying P2P protocol, thearchitecture of CompuP2P is generalized enough to be builton top of other structured P2P networks, such as CAN [6].We provide a brief introduction to the Chord protocol inAppendix C.

GUPTA ET AL.: COMPUP2P: AN ARCHITECTURE FOR INTERNET COMPUTING USING PEER-TO-PEER NETWORKS 1307

3.2 Resource Trading Layer

As shown in Fig. 1, the functionality of this layer can befurther divided into three sublayers:

. Market lookup protocol: It ensure that sellers andbuyers looking to trade a commodity converge onthe same market.

. Resource pricing protocol: The pricing mechanismensure that both sellers and MOs are suitablycompensated for the service they provide to clients.

. Dynamic market creation protocol: It is used forselecting nodes that act as MOs for specificcommodities. The protocol is robust against MOsfailing and new nodes joining the system.

3.3 Service Layer

The service layer accept service requests from a user. Aservice can be a computation task or a data storage request.A computation task is submitted by a user in the form of anXML task file. The task file is parsed and appropriatecomputing nodes in the network are determined that canexecute the associated subtasks. The service layer allows auser to specify reliability and timeliness requirements onthe result of computation, while accessing the systemresources. Moreover, a user interested in backing up thelocal data can also request the service layer to search forappropriate storage nodes in the network. The data isusually replicated at multiple remote nodes and is stored ineither plain-text or encrypted format.

3.4 CompuP2P Usage Steps

To demonstrate the working of the overall system, webriefly describe the steps taken by a user to carry out adistributed computation.

1. The user submits an XML formatted task file to theservice layer. The task file specifies for each subtaskvarious attributes, such as input parameters, CPUcycles required, maximum offered price for success-ful execution, etc.

2. The service layer parses the task file and queries theresource trading layer for the appropriated sellernodes.

3. The resource trading layer looks up the market(s)that trade in resources required for the successfulcompletion of the subtask(s).

4. The MO node is queried about the available sellers.The MO acts as a matchmaker for finding the bestseller, for example, the seller that offers the neededcompute power at minimum cost. The MO iscompensated based on the marginal cost of thesellers listed in the market.

5. The resource trading layer returns the information,like the IP address of a selected seller, to the servicelayer.

6. The service layer submits the subtask to the seller.Depending on a user’s requirements, the servicelayer also provide various fault-tolerance featuressuch as checkpointing and replicated computing.The seller node is compensated after the computa-tion is completed.

7. Finally, the user is notified of the execution resultswith the output(s) being stored in appropriate file(s),as specified by the user in the task file for thesubtask.

4 RESOURCE TRADING

At the heart of CompuP2P is its resource trading layer. Thislayer is responsible for creation and management of markets.For concreteness, here we use compute power as the resourceunder consideration. However, the proposed mechanisms forresource trading are equally applicable to other resourcetypes, such as disk space, memory, bandwidth, etc.

Each node based on its current and past load estimates theaverage number of resources that would remain idle infuture, for example, on Unix platform a user can usecommands “top” and “uptime.” Suppose a node determines


Fig. 1. (a) CompuP2P: system architecture. (b) A conceptual view of CompuP2P.

that it has C cycles/sec. available for the next T time units

(where T is some large enough time period) that it can

provide or make available to others for processing. In case

some other resource, say disk space, is under consideration

then we would use another appropriate unit likeG gigabytes

for T time units. It must be noted that the same value of

number of CPU cycles/sec. might represent different

amounts of compute power for different nodes. This might

happen if nodes have different hardware and/or software

configurations. We use the unit of cycles/sec. to represent

normalized equivalent amounts of compute power at

different nodes in a heterogeneous system.Once the amount of idle compute power has been

estimated, the next step is to determine how to sell them.

Moreover, buyers needing extra compute power should be

able to locate the right sellers and purchase the needed CPU

cycles from them. The related and equally important issue

is how the sellers should price their CPU cycles in order to

maximize their profits. We address these issues in the

following sections.

4.1 Constructing Resource Markets

Since different nodes have different amounts of compute

power to sell and purchase, it is necessary to create suitable

markets to permit buyers and sellers to come together andtrade the amount of compute power they require. For abuyer to sequentially search the entire network for the bestavailable deal is a very time consuming and expensiveoperation. Also, selecting one node, say the successor ofChord ID zero, for trading all available compute power inthe network is not a good idea either. This is because relyingon one node can lead to extreme scalability, fault-tolerance,and security problems.

For efficient creation and lookup of compute powermarkets, we propose two schemes that attempt to uniformlydistribute the location of and responsibility for maintainingthose markets across the network. Both the schemes useChord for market assignment and lookup, however, theydiffer from each other in the overhead involved and themanner in which nodes are selected for running markets forvarious commodities. The term commodity as used hererepresents a range of idle CPU cycles/sec. values. Eachmarket deals in only one type of commodity (i.e., homo-geneous markets). A single physical node may be respon-sible, i.e., be a market owner ðMOÞ, for more than onemarket.

Fig. 3 depicts how nodes with different values of idlecompute power C join different markets. Although, forsimplicity of discussion, we have used C as a discrete value,in actual practice it refers to a well-defined range of valueswithin which a node’s idle processing capacity can lie.Thus, nodes with different but close enough idle processingcapacities trade in the same market.

We describe below two schemes for the creation ofcompute power markets.

4.1.1 Single Overlay Scheme

In this scheme, the value C computed by a seller acts as theChord ID for locating the corresponding compute powermarket. The successor node of Chord ID C is assigned theresponsibility for maintaining the market for that particularidle compute power. It is possible that several computepower values map to a single node and then that node isresponsible for running different markets, all dealing indifferent commodities.

This scheme is very simple to implement and involvesnot much additional overhead. Compute power markets are


Fig. 2. Mapping of keys to nodes in Chord.

Fig. 3. Creation of markets for CPU cycles in CompuP2P.

searched using the normal Chord lookup protocol. In otherwords, if a node needs to purchase x cycles/sec., it simplylooks up for the market maintained by the successor ofChord ID x. The drawback of this scheme is that if the idlecompute power values in the network happen to be in avery narrow range, then most of the markets would map toonly a very few distinct physical nodes. Those nodes canthen become the bottleneck and degrade the systemperformance. Moreover, search for a suitable market by abuyer might potentially require several attempts. In eachattempt, the amount of compute power searched for issuccessively increased, until a desired seller with adequatecapacity is discovered.

4.1.2 Processor Overlay Scheme

In order to more uniformly distribute the responsibility forrunning the compute power markets and to bound the searchtime for an appropriate seller, an additional overlay can bemaintained that keeps information about available idlecompute power at different sellers in the network. All MOs,which are responsible for various commodities, constitutethis Chord-based overlay network. The total ID space of thisnew overlay is equal to the maximum amount of computepower that may possibly be available on any single node andis upper-bounded by 2c � 1, where c is a constant andrepresents the number of bits used to represent the maximumvalue of idle CPU cycles/sec. We assume that the value of c islarge enough to represent the idle processing power of even avery large computer system.

The process of selecting a MO for a commodity isillustrated in Fig. 4. It depicts an existing Chord networkthat is comprised of all the nodes, and m is the Chord IDsize in terms of the number of bits. A node on determiningits value for C applies a hash function to C to find thecorresponding Chord ID ð¼ hashðCÞ, a value between 0 and2m � 1. The successor node of hashðCÞ is then the MO forthe market trading in commodity C. The various MOsdefined in this manner then together form another overlaynetwork, called the processor overlay, which has ID spacefrom 0 to 2c � 1. The ID of a MO in this new overlaynetwork is simply the value C whose hash value wasmapped to it in the initial Chord network. Stated otherwise,the ID of a MO in the processor overlay network, calledCPU Market ID (CMID), is the number of CPU cycles/sec.that are being sold in its market.

It must be noted that in the above description, it ispossible that a single node in the initial overlay network isthe MO for several different markets, causing it to havemultiple CMIDs assigned to it in the processor overlaynetwork. Each CMID value is represented by a differentnode in the processor overlay, as shown in Fig. 4. MOs (i.e.,nodes comprised of the processor overlay) periodicallysend out a broadcast message identifying themselves tonodes in the network. Each node needs only storeinformation about a single MO (or at most a few for faulttolerance) to be able to perform lookups in the processoroverlay.

The lookup in processor overlay, requires 12 ðlogMÞ steps

on average, where M is the number of different markets.Moreover, nodes store OðlogMÞ routing information tosupport the Chord protocol. These results are due to theproperty of the Chord protocol.

The search mechanism for the compute power in processoroverlay is performed based on the number of CPU cycles/sec.(which acts as the lookup key) that a client requires forprocessing. The client first contacts any of the known MOsand forwards the lookup request to it. The selected MOsearches for an appropriate market for the desired computepower in the processor overlay network. The lookup processfinally returns the IP address of theMO that runs the marketfor that compute power or the nearest higher compute powervalue available in the network. For example, if only twocompute power markets (with commodity values b and c)exist in the network, and a client desiresa (wherea < b < c),then the above mechanism returns market for b instead of c.The MO is then contacted to obtain information about thesellers listed in the market.

4.2 Resource Pricing

Pricing is nontrivial when there are either multiple at parsellers from a buyer’s point of view or when a buyer istrying to minimize its cost of processing (again assumingmultiple sellers). Since the sellers in CompuP2P areindependent selfish agents, they try to price their idlecompute power in a manner that maximize their profits.Utilizing the model that a transaction involving the tradingof compute power can be modeled as a one-shot game andusing the results from game theory and microeconomics(the classical Prisoner’s dilemma problem [9] and Bertrandoligopoly [12], respectively), we can see that long-termcollusion among compute power sellers (and MO) is


Fig. 4. Processor overlay schema using the CPU capacity values given in Fig. 3.

unlikely to occur. One-shot model of a compute powertransaction is reasonable to assume, since once a seller sellsits compute power, it delists itself from the market andperhaps move to another market for selling its remainingcompute power, if available. Moreover, in a dynamicsystem, where nodes continually join and leave the net-work, it is difficult to keep track of nodes that do not fulfilltheir collusion agreements. Thus, nodes are not likely to bepenalized based on their past behavior.

In one-shot Prisoner’s dilemma game, noncooperation isthe only unique Nash equilibrium strategy for the players. Infact, the model of Bertrand oligopoly suggests that sellers(irrespective of their number) would not be able to chargemore than their marginal costs for selling their resources (see[9] for a game-theoretic derivation of this result). In Bertrandoligopoloy, sellers strategy is to set “prices” (as opposed to“outputs” in Cournot oligopoly), and is thus more reasonableto assume in the context of CompuP2P.1 In CompuP2P, all thesellers in a market sell the same amount of a computingresource. As a consequence, sellers, irrespective of how manythere are in a market, in CompuP2P set prices equal to theirmarginal costs only. The marginal cost of providing acomputing resource can include among other things—listingprice, bandwidth cost for message exchange, etc., and isrepresented by MCi for a node, i.

4.2.1 Providing Incentives to Sellers

Since the best pricing strategy for sellers is to charge equalto their marginal costs, it results in zero profits for them.Therefore, sellers would not be motivated to sell theircomputing resources unless some other incentive mechan-isms are devised for them. Below, we describe two suchstrategies depending on whether fixed or variable listingpricing is used to compensate a MO. Fixed listing pricing islike a flat subscription charge that a MO charges to thesellers listed in the market. On the other hand, in variablelisting, the compensation that a MO receives is dependenton the dynamics of the market, for example, the MOreceives a certain cut from each transaction that takes placein the market.

Strategy For Fixed Listing Pricing. If fixed listing pricingis possible, then a MO has no incentive to cheat and, thus,we can use the technique employed in Vickrey auction [11].A seller, when it joins a market, provides its marginal costinformation to the MO. A buyer, looking to minimize itscost, selects the seller with the least marginal cost, but theamount it has to pay to the seller is equal to the secondlowest marginal cost value listed in the market. Thisselection scheme is called reverse Vickrey auction.

The above strategy provides nonzero profit to the selectedseller and ensure that sellers state their correct marginal coststo theMO (see [11] for the truth-eliciting property of Vickreyauction). The strategy is also inherently secure because even ifsellers learn about the posted marginal costs, they cannot takeundue advantage of that information to post a lower marginalcost than their actual values. To understand this, consider thefollowing simple example.

Example. Suppose seller A has the marginal cost ðMCAÞ of 5and the lowest marginal cost among all the sellers

different from A ð¼MCA�1Þ is 4. If A hides its true MC

and posts it as 3 in order to get selected, its actual payoffwould be ðMC�1

A �MCAÞ or 4� 5 ¼ �1, i.e., it wouldsuffer a loss of �1. Thus, it can be seen that the onlyrational strategy for a seller is to post its correct MC. Inthis incentive scheme, a seller A selected for processingmakes a profit of ðMC�1

A �MCAÞ.

Strategy For Variable Listing Pricing. If variable listingpricing is being used, the above scheme based on Vickreyauction cannot be employed. This is because Vickreyauction is designed to be used by nonselfish auctioneers(here, MO is the auctioneer), whose goals are to maximizesystem efficiency as opposed to personal gains. Whereas, invariable listing pricing, a MO has an incentive to behaveselfishly to maximize its profits. For the case of fixed listingpricing, this selfishness was not a problem, since the payoffthat a MO received was fixed. But, if the payoff that a MOreceives is dependent on a transaction outcome, then it hasincentive to cheat. To understand how an MO may cheatconsider the following example.

Example. Let us say, a MO receives 10 percent of atransaction value from the sellers. Suppose there arethree sellers, A, B, and C, currently listed in the market.The marginal costs of A, B, and C are 100, 200, and 300,respectively. If a buyer now makes a request for thelowest cost supplier, then the MO has incentive to reportC as the lowest cost supplier, instead of A. This isbecause by doing so the MO earns a profit of 30 ð¼300�10=100Þ instead of 10 ð¼ 100�10=100Þ. Even if Vick-rey auction is used, the MO has incentive to report 200and 300, instead of 100 and 200 as the lowest and secondlowest cost values, respectively, to the buyer.

In order to deal with the selfish MO problem, wepropose a max-min payoff strategy. This strategy makes thepayoff to a seller and MO complementary to each other, i.e.,if the seller receives a high payoff, then the MO receives alow payoff, and vice versa. We define the following simplemodel for this strategy. Let there be S sellers in a market,represented by 1; 2; . . . ; S, such that MCi < MCiþ1 for all1 � i � S � 1. The sellers are not aware of each other (andof the buyers) and only know their own marginal costs,which they truthfully report to the MO. Buyers are alsocompletely unaware about the sellers that are listed in themarket and rely on the MO to give them information aboutthe lowest cost supplier.

The payoffs to the MO and the selected seller by thebuyer under the max-min payoff strategy (based on themarginal cost values that a buyer receive from the MO) areas follows:

PayoffMO ¼ ðMC0S �MC01Þ=ðMC0SÞ2 þ �

Payoffseller ¼MC01 þ 1:ð1Þ

MC01 and MC0S in the above equation refer to themarginal cost values of the lowest and highest cost supplier,respectively, as reported by the MO to the buyer. Note thata MO can manipulate the reported values if doing soincreases its payoff. Also, � is a fixed payoff that a MO

receives from a buyer irrespective of the marginal costs ofthe sellers. Therefore, the max-min payoff strategy can be


1. In the Appendix, we provide an introduction to the game theoreticideas used in this paper.

considered to implement a hybrid listing pricing that hasfeatures of both fixed as well as variable listing pricing.

The above payoff values guarantee that the total cost tothe buyer is bounded, and the best strategy for the MO is toreturn the lowest cost supplier only. We formalize this inthe form of the following lemma.

Lemma 1. Assuming one-shot model of compute powertransactions, the payoff strategy in (1) guarantees thefollowing:

1. The lowest cost supplier is always selected.2. The payoff received by the selected seller covers its

marginal cost of providing the service.3. The total cost to the buyer is bounded.4. The payoff to the MO is variable depending on the

dynamics of a market, specifically, it depends on themarginal costs of the sellers listed in the market.

Proof.

1. The MO can increase its payoff by reporting a lowvalue for the lowest listed marginal cost, i.e.,minimizing MC01 as much as possible. However,MC01 cannot be decreased below MC1, the truelowest marginal cost since, otherwise, the seller(here, seller 1) gets a payoff of MC01 þ 1ð�MC1Þ.Since a seller does not provide its service unlessits payoff is greater than its marginal cost, the beststrategy for the MO is to set MC01 ¼MC1 andreturn the lowest cost supplier for processing.

2. This is implied from (1) where we see that thepayoff received by the seller is one more than itsmarginal cost.

3. From (1), the payoff to the MO is maximized forMC0S ¼ 2 �MC1 (after setting @PayoffMO

@MCS¼ 0Þ, giving

it a payoff of 1=ð4 �MC1Þ. Note that in the givennetwork model, it is difficult for a buyer to verifythe marginal cost values it receives from the MO.Thus, the total cost to the buyer is bounded, andis equal to 1=ð4 �MC1Þ þ � þMC1 þ 1.

4. It follows from the description of the payoffvalues given by (1). tu

Above, we assume that a MO serves the buyers in theorder in which it receive requests from them. Moreover,once a seller has been selected for processing, it delists itselffrom the market, and joins some other market if it hassufficient compute power remaining. It must be noted thatthe max-min payoff strategy is not effective if a MO is oneof the sellers in its own market. To guard against such apossibility, a client can ensure that the IP address of theseller it is contacting for a computation is not the same asthat of the MO that returned the seller as the one with thelowest cost.

5 PROTOTYPE IMPLEMENTATION OF COMPUP2P

We have implemented a Java-based prototype of theproposed CompuP2P architecture for the sharing ofcompute power, and have deployed it in our lab forrunning compute intensive simulations. Java, owing to itsplatform independence and write-once run-anywhere fea-ture, enables easy migration of tasks from one node toanother in a heterogeneous system.

Screen-snapshots of the implemented CompuP2P proto-type, as it appears to a user, are shown in Figs. 5a and 5b.The first tab, “Usage Policy,” allows a user to specify theusage constraints on the local shared resources. Forexample, a user can specify the CPU load levels (in


Fig. 5. (a) Screen snapshot of Usage Policy tab. (b) Screen snapshot of Resource Trading tab.

percentage) beyond which the node is not allowed to shareits compute power. To prevent a task from running forever,user can also impose the maximum allowable runtime ontasks received for execution. The user can specify specifictimes of the day compute power can be shared. Forexample, one can specify that compute power can beshared only during nighttime when the machine is mostlyunutilized. Moreover, for sharing storage space, the usercan specify the total allocated shared space, along with thedirectory name where the received files are to be stored.Furthermore, the user can limit access to the machine byspecifying the IP addresses of nodes that are not permittedto utilize the shared resources.

The second tab, “Resource Sharing,” is divided into twocomponents. In the first component, the user specifieswhether compute power and disk storage are shareable ornot (usage policies described above are consulted before theresources are actually made shareable). The second compo-nent lets the user advertise files, which can be downloadedby others in the network.

The “File Storage” tab allows a user to back up its localdata on multiple remote machines in the network.

A user submits its task to the service layer in the form ofa task file. The task file contains a description of varioussubtasks (a given task is assumed to be broken into severalindependent subtasks) that need to be solved. A sampletask file is shown in Fig. 6. For each subtask, the followinginformation is included:

. Code ID (or name) of the executable file for thesubtask. The executable file (if not locally available)can be downloaded either from a well-defined codeserver or can be searched for and downloaded just asother normal data using code ID (or name) as thekey. To ensure security, the computing node

executes the downloaded code in an appropriatesand-boxing environment. A subtask is executed at asingle node and, thus, define the level of granularityat which parallelism can be achieved.

. Names of input and output files to be used. If theinput files are not available with a computing node,they can be searched for using the Chord lookupprotocol.

. Estimated amount of compute power required.

. User’s budget, i.e., the maximum amount of rewardthat a user can give in order to get the subtasksuccessfully executed.

. An indication whether the subtask is to be periodi-cally checkpointed or not, and the estimated size ofcheckpoint data.

. Platform specifications desired for selecting a seller.

The submitted task file is parsed and a thread isspawned for each subtask. The thread created is responsiblefor looking up an appropriate seller node, negotiating theprice for subtask execution, and finally obtaining the resultsof computation and storing them in the output file(s)specified by the user. The task file is supplied to the servicelayer via the “Task Submission” tab.

A node first enters the network by contacting a bootstrapserver running at a well-known IP address and portnumber. This bootstrap server is referred to as AdminServerin our implementation. AdminServer has information aboutall live nodes in the network, and returns the (IP addressand port number) of a randomly selected existing nodewhen contacted by a new node. The new node then usesthis returned value to join the Chord network and updateits routing table. It must be noted that the continuousavailability of AdminServer is not critical for the successfulrunning of the CompuP2P system. Nodes that are alreadypart of the system are completely unaffected by its failure.Whereas nodes that are leaving the system can cache theIP addresses of the nodes that are in their Chord routingtables, so that when they again join they do not have to relyon AdminServer for finding a bootstrap node.

In our current implementation, we use the KARMAprotocol [22] for maintaining nodes’ accounts in the system.KARMA is a completely distributed protocol for imple-menting a system of virtual currency in P2P networks. Theprotocol ensures secure transaction between any pair ofnodes. By secure, we mean a node cannot falsely increase its(and reduce others’) currency, and also does not stand togain from a transaction (process where nodes provideservice for some compensation) unless it successfullycommits its half of the transaction. We provide a briefintroduction to the KARMA protocol in the appendix. Fordetails, one can refer to [22]. When a node first enters thenetwork, it is assigned some minimum currency that iscredited to its account. Users’ accounts are automaticallydebited (credited) whenever they buy (sell) compute poweras per the pricing strategy outlined in Section 4.2. Buyerswith insufficient balance are not permitted to use comput-ing power of others in the network.

In addition to the GUI-based interface, one can use ourRemoteExecution API for submitting a task file to theCompuP2P system. We have provided a TCP/IP socket


Fig. 6. XML-based task file.

interface for allowing the RemoteExecution API, which is inJava, to be usable by applications written in otherlanguages. Applications supply the task file name overthe socket connection, and are provided a notification (ofsuccess or failure) when all the subtasks defined in the taskfile are finished executing.

The system is clearly intended for very coarse-grainedparallelism. The efficiency is mostly determined by the ratiobetween the computation time of subtasks to the commu-nication effort needed to send them and handle theoverhead. To achieve high efficiency, subtasks should berelatively heavy in terms of computation time.

In order to address the nodes’ heterogeneity problem,there are at least a couple of possible alternatives. Onealternative is to use the SPECjvm98 benchmark [14]. Abenchmark program can be selected based on the type ofapplications typically submitted by the users of the network.Benchmarks help to normalize the compute power values sothat a given value is interpreted similarly by all the differentnodes. Another alternative that is currently used in Com-puP2P is to use anMO as a matchmaker. AnMOnow stores thedetailed platform description of all the sellers in the market.This description includes information such as OS type, OSversion, processor configuration, etc. On receiving a requestfrom a buyer, the MO selects the seller that not only has thelowest cost, but also meet the platform specifications asdesired by a client for its subtasks.

5.1 Handling Failures of MOs and Listed Sellers

It is possible that nodes selected as MOs as well as sellerslisted in those markets might fail. To account for suchpossibilities, we incorporate the following additionalstrategies in our prototype implementation:

1. Sellers periodically relist themselves in a (new)market. This is done irrespective of whether theamount of a computing resource they are offeringhas changed or not. This periodic listing takes care ofthe following two problems that may arise in anydynamic system—1) MOs leaving the network and2) new nodes joining the network that may replacesome existing MOs as the new MOs for therespective markets.

2. Likewise, everyMO periodically purges the informa-tion it maintains about the listed sellers. Thus, sellerinformation is maintained as a soft-state information,and is never outdated for too long. Additionally, anMO before returning information about the selectedseller to the buyer checks whether the seller is stillalive (i.e., is part of the network) or not.

5.2 Handling Failures of Buyers and Sellersduring Computations

It is possible that the client or the computing node may failbefore the computation is complete and the seller hasreceived the results. In our prototype implementation, if thecomputing node goes down, then it loses all state informa-tion associated with the computation, and so it forgoes theright to demand any compensation from the client.However, if the client goes down, then it reads its log filecontaining information on all previous computations it

requested, and see the ones that did not complete before itfailed. The log file has information about the computing andstorage node for a particular computation. The buyer canattempt to contact the computing node and check if thecomputation is complete or not. If the computation is stillnot over, the computing node would never know that theclient had failed. On the other hand, if the computation isalready over the computing node discard the results after acertain time period of contacting the client and detectingthat it is not online.

5.3 Fault-Tolerant Computing

It is possible that a computing node is not able to finish thecomputation assigned to it, either because it leaves thenetwork, crashes, or the computation takes longer tocomplete than initially anticipated by a client. Under suchcircumstances, it may be expensive to restart the computa-tion all over again. To handle such cases, it is useful toperiodically checkpoint the computing node’s state, so thatif required the failed computation can be migrated toanother node in the network.

Unlike traditional checkpointing, which relies on dedi-cated checkpoint servers to store the processing state, wepropose to use server-less checkpointing in which nodes thatstore the checkpoint data are determined on-the-fly. Similarto the techniques outlined in Section 4.1 for the sharing ofcompute power, we can construct markets for memorystorage. The client based on its estimation of the amount ofcheckpoint data can reserve the required memory space.The nodes performing computation are made aware of suchnodes, to which they periodically send a checkpoint of theircomputations. Upon failure of a computing node, the storedcheckpoint data can be used to restart the computation atanother suitable node in the network. We use the objectserialization feature provided in Java, which enables afailed computation to be continued at a different node uponfailure of an initially allocated processing node. Thecheckpointing protocol built into CompuP2P is illustratedby Fig. 7.

The service layer at a user node selects a computing nodeto which it sends the task for execution, and at the sametime selects a storage node that has sufficient disk capacityto store the periodic checkpoints generated by the task. Thestorage node is the checkpoint server for the task inconsideration, and is selected based on the size of thecheckpoint data produced by the task. The size value isspecified by the user and is equal to the total size of all theobjects that are needed to restart the task. Our serverlesscheckpointing protocol is designed to take into account thefailure of both the computing as well as the storage nodes.Fig. 7 describes the steps followed, in CompuP2P, toimplement serverless checkpointing. In our experience withthe CompuP2P implementation, we have found that all longrunning computations (even those up to 30 minutes long)finish successfully as long as there are sufficient computingand storage nodes in the network. This result is observedeven for a high churn rate (rate at which nodes leave andjoin the network), say 1 node joining or failing every1 second in a 20 node system, in the system. One of the mainreasons for the robustness of the serverless-checkpointing


protocol is the fact that the stabilization procedure of the

underlying Chord protocol is extremely fast and efficient.Similarly, a user who purchased memory storage from a

seller in the network might want to be sure that the seller

actually has stored the file. In this case, our system allows

the user to periodically ask the seller to send some parts of

the file. Here, we also allow the user to store an encrypted

version of the data with the seller.Further, in practice, errors in computation and/or

communication of results can occur. Computation errors

can occur due to faulty software/hardware at the comput-

ing node, or when a malicious node deliberately produces

incorrect output. Such errors might be hard to detect and

correct. To increase the reliability in the correctness of the

end results, the following alternatives can be used:

. Redundant computations, as also used in SETI@Home [3], can be employed. Basically, this schemeinvolves performing the same computation multipletimes at different nodes and then selecting the resultproduced by the maximum number of computingnodes. This capability is available to the userthrough the “Task Submission” tab in our Com-puP2P GUI. Also, the user is notified about thediscrepancies in the results obtained along with theIP addresses of nodes that generated those results.Note that redundant computations, like serverlesscheckpointing, also serve to address the problem ofcomputing nodes failing or leaving the network.However, for long-running tasks checkpointingprovides a better guarantee on the timeliness of thefinal result(s). We believe that best performance isachieved (although at a higher cost) by combiningboth these fault-tolerance strategies.

. The tasks may be designed in a way that certaincharacteristics of the answer are known in advanceto the client, but hard to deduce just from the taskcode. In these cases, an answer that has thesecharacteristics may be assumed correct.

. Some tasks may return answers that are easilyverified correct. For example, a task, which solvesan equation using some complex method, may beeasily verified by plugging the solution into theequation.

However, all the fault-tolerance features come at anincreased cost to a user. The user’s budget should besufficient to cover the cost of reserving memory space tostore the checkpoint data and/or compensate the redun-dant computing nodes for their processing.

5.4 CompuP2P Applications

We list here several applications of CompuP2P system.These applications are ones which can utilize the idlecomputing power in P2P systems. A common characteristicof all these applications is that they are all loosely coupled,i.e., they can be broken into rather independent subtasks,each heavy in terms of processing power requirements, butrelatively light in terms of communication requirements.Some such applications are brute force search, code breaking,simulated annealing, and Monte Carlo simulations. All of theseapplications primarily involve generating many solutions inparallel and then using the solutions to come up with ananswer for the initial problem.

We have found CompuP2P to be very useful for runninglarge simulations. We used CompuP2P for running parallelsimulations on multiple optical network topologies asgenerated by a user using ISTOS [15], which is an advancedtool for simulating fiber optic networks. Users in ISTOS cancreate multiple network topologies on which simulationsare to be carried out. Earlier, all simulations were sequen-tially executed on a single back-end server. (More informa-tion on ISTOS can be found in [15].) However, now we canexploit the CompuP2P architecture to distribute the task ofsimulating different network topologies to different nodesin the network which agree to share their idle computingpower. It was observed that the resulting system provided atremendous speedup in terms of the time required to finishsimulation which runs on all the submitted topologies. The


Fig. 7. (a) Serverless checkpointing in CompuP2P. (b) Checkpointing protocol steps.

resulting speedup was very close to the number of differentnodes which were used in parallel to run the simulations.This high speedup was possible mainly because of the lowcommunication overhead—input to the computing nodesinclude a topology specification file and other parametersneeded to start a simulation, and output include a filecontaining the simulation results (no communicationamong the computing nodes is required). (An executablefile of the simulation code was preinstalled on the nodes).

Fig. 8 shows the speedup achieved as a result of usingthe RemoteExecution API. In our experiments, the number ofcomputing nodes were 10, and the number of subtasks weresuccessively increased from 1 to 10. It can be observed thatCompuP2P provides substantial tremendous gain and thisis achieved by simply utilizing the idle capacity of machinesin the network. The speedup would become close to 1 as thetask computation time is increased. Ups and downs seen inFig. 8 for the parallel execution time are due to theheterogeneity of processors used in the experiment.

As a demonstration of the effectiveness of CompuP2P,we have attempted to solve some other well-knownproblems, such as the travelling salesman problem, usingCompuP2P.

5.4.1 Traveling Salesman Problem

In the traveling salesman problem (TSP), there are N citiesand the distances from city i to city j is dij. A tour is a paththat starts from a city, visits each city exactly once, and goesback to the starting city. The goal is to find a tour withminimum tour length. Genetic algorithms (GAs) have beenused to solve well-known NP-hard combinatorial problemsand optimization problems such as TSP.

GAs are adaptive heuristic search algorithms based onthe evolutionary ideas of natural selection and genetics. Assuch, they represent an intelligent exploitation of a randomsearch used to solve optimization problems. Briefly, GAssimulate the survival of the fittest among individuals overconsecutive generation for solving a problem. Each genera-tion consists of a population of character strings that areanalogous to the chromosomes that we see in humanDNAs. Each individual represents a point in a search spaceand a possible solution. The individuals in the populationare then made to go through a process of evolution.

We use GAs to solve TSP for measuring the speedupprovided by CompuP2P when running the SequentialGenetic algorithm (SGA) and Parallel Genetic algorithm(PGA). We used the code from GATSS [16], and made some

modifications to it for our experiments. A 52 city symmetricðdij ¼ djiÞ Euclidean TSP instance was chosen as a test casefrom TSPLIB [17].

For our experiments, in SGA, we chose an initialpopulation (P) of 1,000, 2,000, 3,000, and 4,000 tours(chromosomes), and the simulation stops when a bettersolution is not reached for 500 iterations (generations in GAterminology). To test SGA, a total of 20 runs wereconducted.

The simplest way of parallelizing a GA is to executemultiple copies of the same SGA, each one on differentnodes. On CompuP2P, five compute nodes were chosen andon each processor, SGA starts with different initial sub-population (P/5 chromosomes), evolves and stops inde-pendently. The complete PGA halts when all processorsstop. This approach has two advantages as discussed below:

. No intercommunication between processors due tocoarse grained parallelism.

. Subpopulation diversity reduces the chance that allprocessors prematurely converge to the same poorquality solution.

This approach is equivalent to simply taking the bestsolution after multiple executions of the SGA on differentinitial populations [18]. Fig. 9 gives the average values ofthe execution time for SGA and PGA in seconds. Thespeedup obtained by the execution of PGA using Com-puP2P was approximately 4.3, which is close to the numberof nodes used.

5.5 CompuP2P Overhead

CompuP2P is a lightweight architecture and incurs minimaloverhead on the system. It is built on top of Chord, which is ascalable, efficient, and robust protocol [5]. In this section, weexamine in detail the additional overhead incurred by thesellers, buyers, andMOs by the CompuP2P architecture. Theoverhead is in the form of either message communication orstate maintained by each of these entities.

1. Message communication: As discussed in Section 4.1,messaging overhead incurred by both buyers andsellers to locate a market is OðlogNÞ in case of singleoverlay scheme, and OðlogMÞ (where M is thenumber of different markets) for processor overlayscheme. Once a buyer has selected a seller forservice, further communication between them takesplace using a direct TCP/IP connection, bypassingChord routing. Message communication overheadincurred by MOs is almost negligible (apart fromdirect TCP/IP connections with buyers and sellers).

2. State maintenance: In the processor overlay scheme,all nodes have to maintain additional (apart from theinitial Chord-based overlay state) OðlogMÞ routinginformation for maintaining the processor overlay,and in single overlay scheme, no additional routinginformation is required.

A buyer (seller) maintains the IP address of a seller(buyer). Moreover, information maintained by MOs isminimal. This information size, S, is given as follows:

S ¼ n � a � s: ð2Þ


Fig. 8. Figure showing the total time required to execute subtasks using

CompuP2P, as opposed to the time required when all the computations

are carried out on a single node.

Here, n is the number of sellers in a market, a is thenumber of different attributes of a seller, and s is the spacerequired to store a value of each attribute. To see how muchthe value of S evaluates to, let us consider a MO that storesinformation about 10,000 sellers. The MO might storeseveral attributes pertaining to a seller. These attributesinclude information regarding a seller’s IP address, margin-al cost, OS type, OS version, processor configuration, etc.Suppose that there are 10 attributes value for each seller,and each attribute require 4 bytes of memory space. Then,the total information maintained by the MO is equal to,10; 000 � 10 � 4 � 400 KB. Thus, even for a large-sizedmarket, its state information (< 0.5 MB) can easily residein any modern PC’s RAM, which are typically 512 MB.Furthermore, since the entire market information easily fitsinto a MO’s main memory, lookups to select an appropriate(based on a client’s request) seller are also very fast.

6 RELATED DISTRIBUTED COMPUTING PROJECTS

CompuP2P is an architecture for enabling Internet comput-ing, and is thus significantly different from large-scaledistributed computing projects that have been implementedin the arena of grid or public resource sharing computing.These differences were highlighted in Section 2. Here, wecompare CompuP2P with some specific well-known pro-jects, such as Condor [20], Entropia [13], SETI@home [3],and POPCORN [19], to bring out the novelty and usefulnessof this new architecture.

Condor is designed to harness the idle CPU cycles ofworkstations, desktops, servers, etc. Users submit their sets ofserial or parallel tasks to Condor in form of jobs. The Condormatchmaker decides where to run them based on job needs,machine capabilities, and usage policies. Task management iscentralized to ensure that jobs are executed based on thespecified requirements of provider and consumer.

Entropia is a commercial product, and is sold as part ofEntropia’s DCGrid enterprise solution (www.entropia.com). Since the majority of desktops are Windows x86machines, Entropia focuses purely on providing a Windowsx86-based solution, supporting three generations of Win-dows operating systems NT, 2000, and XP. The Entropia

system architecture is composed of three separate layers. Atthe bottom is the Physical Node Management layer thatprovides basic communication and naming, security,resource management, and application control. On top ofthis layer is the Resource Scheduling layer that providesresource matching, scheduling, and fault tolerance. Userscan interact directly with the Resource Scheduling layerthrough the available APIs or, alternatively, users canaccess the system through the Job Management layer thatprovides management facilities for handling large numberof computations and files.

Unlike Condor and Entropia, CompuP2P is completelydecentralized, in the sense that there is no centralized entitythat monitors system state and assign (sub)tasks accord-ingly. CompuP2P uses microeconomic principles andgame-theoretic ideas to govern trading and allocation ofcompute power to tasks. Moreover, CompuP2P is imple-mented using Java and can theoretically run on virtuallyany system. Entropia, on the other hand, is designed forWindows-based system only.

In SETI@home, only one central node can allocate tasksto others, whereas in CompuP2P, all the grid nodes canpurchase compute power and distribute their workload toother machines.

POPCORN provides an infrastructure for globally dis-tributed computation over the whole Internet and uses amarket-based mechanism to trade CPU cycles. However,unlike in CompuP2P, POPCORN uses a trusted centralizedmarket that serves as a matchmaker between the seller andbuyer nodes.

Sharing of CPU cycles in CompuP2P is completelydistributed and fault-tolerant as compared to the schemeproposed in [21] that uses a centralized auction.

7 CONCLUSION

In this paper, we have discussed CompuP2P, which enablesInternet computing. CompuP2P is significantly differentfrom other large-scale distributed computing projects whichhave been implemented in the arena of grid or public


Fig. 9. Performance comparison of SGA and PGA.

resource sharing computing. It can be used for buildinglarge Internet computing infrastructures, and can poten-tially reduce the need for expensive processing or storageservers in an enterprise, for example. Users of CompuP2Pcan harness almost unlimited processing capacity of theentire network in a complete distributed manner withoutusing any centralized administrative authority.

As part of our future work, we would design andimplement a workflow engine [23] and integrate it withCompuP2P. Workflows allow dependencies between sub-tasks to be represented in the form of an acyclic graph, andare an important business tool.

APPENDIX A

GAME THEORY INTRODUCTION

Game theory aims to help us understand situations inwhich decision-makers interact. Game theory is a theorythat uses mathematics to analyze situations of conflict andcooperation between the agents’ interests. A game can beviewed as a situation in which:

1. there are at least two agents,2. each agent has a number of possible strategies (or

actions), which it may follow,3. strategies determined by all the players determine

the outcome of the game, and4. each outcome has an associated payoff (utility), one

for each player.

These payoffs represent the value of that outcome to thedifferent agents.

The basic assumption regarding the decision-makingprocess of agents is that they are perfect utility maximizers.This means that each player is capable of and does make acomplete logical analysis of which is the best way to achieveits objective, taking into account that the other agents do thesame. In particular, utility maximizing agents assume utilitymaximizing opponents. It is customary to use the termrational for utility maximizing.

A.1 Nash Equilibrium

Nash equilibrium is a concept that attempts to capture theidea of a solution of a game. It is supposed to convey anotion of “stability” in strategic behavior. Generally, asolution should be regarded as a prescription or aprediction that a certain outcome will result from a playof a given game. More formally, Nash equilibrium isdefined as follows:

Nash equilibrium is an action profile a� with the propertythat no player i can do better by choosing an action differentfrom a�i , given that every other player j adheres to a�j .

2

A.2 Prisoner’s Dilemma

This is probably one of the most famous games. It goes asfollows: Two burglars, Bob and Al, are detained by the policenear the scene of a crime. The officers put the two criminals intwo separate rooms at the police station for questioning. Eachburglar is given the following options—he can defect, whichmeans that he implicates his fellow criminal, or he can

cooperate, which means that he tells the police nothing. Theofficer in charge describes to the burglars the consequences oftheir choices as follows. If both men cooperate, both will servea reduced sentence of one year in prison for the lesser chargeof carrying a concealed weapon. If both defect and implicateeach other, both will go to prison for five years. But, if oneburglar defects while the other cooperates, the defector willgo free while the cooperator will spend 10 years in theslammer. The strategies in this case are C (cooperate) and D(defect). The payoffs, corresponding “inversely” to the jailsentences, can be given in a bimatrix form as shown in Fig. 10.

The row corresponds to one of the players, say Bob, andthe column correspond to the other player, Al. If, forexample, Bob plays C and Al plays D, then Bob’s payoff is 0,whereas Al’s payoff is 5. The intriguing feature of PD is thefollowing. Bob has an incentive to play D, because if Alplays C, he will get five rather than three (had he played C),and if Al plays D, he will get one rather than zero (had heplayed C). Thus, regardless of what Al does, Bob is betteroff playing D. Exactly the same argument applies to Al.Therefore, action profile (D, D) is the Nash equilibrium forthe Prisoner’s Dilemma game. (This is precisely thedilemma since both players get a payoff of one instead ofthree that they could have got by cooperating.)

A.3 Bertrand Oligopoly

Oligopoly refers to a model of market state when there iscompetition between a small number of firms (or sellers),though it involves no restriction on the number of firms. InCournot oligopoly, each firm chooses an output; the price isdetermined by the demand for the good in relation to thetotal output produced. On the other hand, in Bertrandoligopoly, each firm chooses a price, and produces enoughoutput to meet the demand it faces, given the prices chosenby all the firms. The economic setting for this model is asfollows: A single good is produced by n firms; each firm canproduce qi units of the good at a cost of CiðqiÞ. The demandis specified by a demand function D. The interpretation ofD is that if the good is available at the price p, then the totalamount demanded is DðpÞ.

Assume that if the firms set different prices, then allconsumers purchase the good from the firm with the lowestprice, which produces enough output to meet this demand.If more than one firm sets the lowest price, all the firmsdoing so share the demand at that price equally. Insummary, the Bertrand’s oligopoly game is the followingstrategic game.

Players: The firms.Actions: Each firm’s set of actions is the set of possible

prices.Preferences: Firm i’s preferences are represented by its

profit, which is equal to piDðpiÞ=m� CiðDðpiÞ=mÞ if firm i isone of m firms setting the lowest price, and equal to zero ifsome firm’s price is lower than pi.


Fig. 10. Tabular representation of the Prisoner’s Dilemma game.

2. The notation a� denotes the Nash equilibrium action profile of all theplayers, such that the Nash equilibrium action of player i in this equilibriumis given by a�i .

For the above game (assuming prices to be a multiple ofsome indivisible unit � and the same unit cost c for eachfirm), it can be shown that there exists a unique Nashequilibrium in which the unit price charged by each firm isequal to c.

APPENDIX B

KARMA PROTOCOL

KARMA is a completely distributed protocol for imple-menting a system of virtual currency in P2P networks. Theprotocol ensures secure transaction between any pair ofnodes. By secure, we mean a node cannot falsely increase its(and reduce others’) currency, and also does not stand togain from a transaction (process where nodes provideservice for some compensation) unless it successfullycommits its half of the transaction.

KARMA maintains all of its internal state in a peer-to-peer distributed hash table (DHT). The bank-set BankA of anode A is a set of k peers that independently maintain thekarma balance of that node. KARMA uses the DHT to mapnodes to bank-sets. The k closest nodes in the identifierspace to the Chord ID of A constitute the bank-set of A.Picking k consecutive hosts for the bank-set allows thesecure routing to the bank-set to be performed efficiently.

Each member of BankA stores the amount of karma inA’s account, signed with A’s private key, as well as atransaction log containing recent payments A has made toother nodes. Signing of the balance by A ensures that thevalue is tamper-resistant. The transaction log acts as proofof A’s payment, and comes into play if the other party in thetransaction does not send A the file for which the paymentwas made. The bank-set corresponding to each node alsostores—1) the last used sequence number, which is part ofthe message sent by a node authorizing its bank-set totransfer karma from its account to the account of some othermember (used to eliminate the possibility of replay attacks),and 2) the current epoch number (for periodic currencyadjustments and ensuring that per-capita karma in thesystem is roughly constant).

The karma transfer between nodes, say nodeA to nodeB,takes place as follows: A first sends to B a signed messageauthorizingBankA to transfer a given amount of karma toB.B forwards this message to its bank-set, who contactBankA inturn. If A has sufficient karma in its account to fund thetransaction, the amount is deducted from A’s account andcredited toB’s account, andB can proceed with the resourcetransfer to A. For details about the security aspect of theKARMA protocol, one can refer to [22].

APPENDIX C

CHORD PROTOCOL

Chord is a distributed lookup protocol to efficiently locate anode that stores a particular resource. It supports just oneoperation: given a key, it determines the node responsible forthat key. Each Chord node has a uniquem-bit identifier (ID),obtained by hashing the node’s IP address. Chord views theIDs as occupying a circular identifier space. Keys are alsomapped into this ID space, by hashing them tom-bit key IDs.We will use the term “key” to refer to both the original key andits image under the hash function, as its meaning will be clearfrom the context. Similarly, the term “node” refers to both the

node and its identifier under the hash function. Chord definesthe node responsible for a key to be the successor of that key’sID. The successor of an ID j is the node with the smallest ID thatis greater than or equal to j (with wrap-around), much as inconsistent hashing [10]. A pictorial representation of a Chordnetwork is given in Fig. 2.

Consistent hashing lets nodes enter and leave the networkwith minimal movement of keys. To maintain correctsuccessor mappings when a node ni joins the network,certain keys previously assigned to ni’s successor becomeassigned to ni. When node ni leaves the network, all ofni’s assigned keys are reassigned to its successor. No otherchanges in the assignment of keys to nodes need occur. Also,consistent hashing does a good job of load balancing keysonto nodes. Intuitively, this follows since the use of anappropriate hash function means that node and keyidentifiers can be treated as independent, uniformly dis-tributed random points on the circle. Consistent hashing isstraightforward to implement, with constant time lookups, ifall the nodes have an up-to-date list of all other nodes.However, such a system does not scale. Chord provides ascalable, distributed version of consistent hashing.

Every Chord node maintains a list of the identities and IPaddresses of its r immediate successors on the Chord ring.The fact that every node knows its own successor meansthat a node can always process a lookup correctly—if thedesired key is between the node and its successor, the latternode is the key’s successor; otherwise, the lookup can beforwarded to the successor, which moves the lookup strictlycloser to its destination.

A new node n learns of its successors when it first joinsthe Chord ring, by asking an existing node to perform alookup for n’s successor; n then asks that successor for itssuccessor list. The r entries in the list provide fault-tolerance—if a node’s immediate successor does notrespond, the node can substitute the second entry in itssuccessor list. All r successors would have to simulta-neously fail in order to disrupt the Chord ring, an event thatcan be made very improbable with modest values of r. Animplementation should use a fixed r, chosen to be 2 log2 Nfor the foreseeable maximum number of nodes N .

Lookups performed only with successor lists require anaverage ofN=2 message exchanges. To reduce the number ofmessages required to OðlogNÞ, each node maintains a fingertable with m entries. The ith entry in the table at node ncontains the identity of the first node that succeeds n by atleast a distance of 2i�1 on the ID circle. Thus, every nodeknows the identities of nodes at power-of-two intervals on theID circle from its own position. A new node initializes itsfinger table by querying an existing node. Existing nodeswhose finger table or successor list entries should refer to thenew node find out about it by periodic lookups.

The following two theorems, given in [5], show thatneither the success nor the performance of Chord lookups islikely to be affected even by massive simultaneous failures.Both theorems assume that the successor list has lengthr ¼ OðlogNÞ. A Chord ring is stable if every node’ssuccessor list is correct.

Theorem 1. In a network that is initially stable, if every node

then fails with probability 1/2, then with high probability, the

search for a successor returns the closest living successor to the

query key.


Theorem 2. In a network that is initially stable, if every nodethen fails with probability 1/2, then the expected time to find asuccessor is OðlogNÞ.

ACKNOWLEDGMENTS

The research reported in this paper was funded in part byJerry R. Junkins Endowment at Iowa State University.

REFERENCES

[1] Kaaza, http://www.kaaza.com, 2006.[2] Gnutella, http://gnutella.wego.com/, 2006.[3] SETI@home, http://setiathome.ssl.berkeley.edu, 2006.[4] http://www-unix.globus.org/toolkit/, Mar. 2005.[5] I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, and H. Balakrish-

nan, “Chord: A Scalable Peer-to-Peer Lookup Protocol for InternetApplications.” Proc. 2001 ACM SIGCOMM Conf., 2001.

[6] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker, “AScalable Content-Addressable Network,” Proc. ACM SIGCOMM,2001.

[7] D.P. Anderson, “Boinc: A System for Public Resource Computingand Storage,” Proc. Fifth IEEE/ACM Int’l Workshop Grid Computing,Nov. 2004.

[8] I. Foster and C. Kesselman, The Grid: Blueprint for a New ComputingInfrastructure, second ed. Morgan Kaufmann, 2004.

[9] M.J. Osborne, A Course in Game Theory. Cambridge, Mass: MITPress, 1994.

[10] D. Karger, E. Lehman, T. Leighton, M. Levine, D. Lewin, and R.Panigrahy, “Consistent Hashing and Random Trees: DistributedCaching Protocols for Relieving Hot Spots on the World WideWeb,” Proc. 29th Ann. ACM Symp. Theory of Computing, pp. 654-663, May 1997.

[11] W. Vickrey, “Counterspeculation, Auctions, and CompetitiveSealed Tenders,” J. Finance, pp. 8-37, 1961.

[12] M.R. Baye, Managerial Economics and Business Strategy, third ed.McGraw Hill, 2000.

[13] A. Chien, B. Calder, S. Elbert, and K. Bhatia, “Entropia:Architecture and Performance of an Enterprise Desktop GridSystem.” J. Parallel and Distributed Computing, vol. 63, pp. 597-610,2003.

[14] “Standard Performance Evaluation Corporation,” SPECjvm98Documentation, Release 1.0, Aug. 1998, http://www.spec.org/osg/jvm98/jvm98/doc/index.html.

[15] http://ecpe.ee.iastate.edu/dcnl/DCNLWEB/Tools/tools_ISTOS.htm, June 2005.

[16] GATSS, Genetic Algorithm Traveling Salesman Problem Solver,http://www.acc.umu.se/~top/travel_information.html, June2005.

[17] TSPLIB, Library of Sample Instances for TSP (Traveling SalesmanProblem), http://www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/index.html, June 2005.

[18] L. Wang, A.A. Maciejewski, H.J. Siegel, and V.P. Roychowdhury,“A Comparitive Study of Five Parallel Genetic Algorithms UsingTraveling Salesman Problem,” Proc. 12th Int’l Parallel ProcessingSymp. and Ninth Symp. Parallel and Distributed Processing, Mar.1998.

[19] N. Nisan, S. London, O. Regev, and N. Camiel, “GloballyDistributed Computation over Internet—The POPCORN Project,”Proc. 18th IEEE Int’l Conf. Distributed Computing Systems, pp. 592-601, May 1998.

[20] P. Wagstrom, An Overview of Condor. Feb. 2002.[21] M. Senior and R. Deters, “Market Structures in Peer Computation

Sharing,” Proc. Second Int’l Conf. Peer-to-Peer Computing (P2P ’02),2002.

[22] V. Vishnumurthy, S. Chandrakumar, and E.G. Sirer, “Karma: ASecure Economic Framework for Peer-to-Peer Resource Sharing,”Proc. Workshop Economics of Peer-to-Peer Systems, 2003.

[23] I. Foster and C. Kesselman, The Grid: Blueprint for a New ComputingInfrastructure, second ed. Morgan Kaufmann, 2004.

Rohit Gupta received the undergraduate de-gree in computer science and engineering in1997 from REC Kurukshetra, India. He receivedthe PhD degree from Iowa State University in2005, and the MBA from the University of SanFrancisco in 2001. He is currently working forAmazon.com in the Transaction Risk Manage-ment Group. From 1997 to 1999, he worked as aresearch engineer in the Call Processing Group,working on ISDN and WLL technologies, at the

Centre for Development of Telematics (C-DOT), New Delhi, India. Hisresearch interests include networking, peer-to-peer systems, paralleland distributed computing, and telecommunications.

Varun Sekhri received the BTech degree incomputer science and technology from IndianInstitute of Technology, Roorkee, in 2001. Hereceived the MS degree in computer engineer-ing from Iowa State University in 2005. In 2004,he worked as a software intern with NokiaResearch Lab, Boston. His research interestsinclude computer algorithms, distributed sys-tems, and network security. He is currentlyworking at Microsoft, Seattle.

Arun K. Somani received the MSEE and PhDdegrees in electrical engineering from the McGillUniversity, Montreal, Canada, in 1983 and 1985,respectively. He is currently the Jerry R. JunkinsEndowed Chair Professor of Electrical andComputer Engineering at Iowa State University.He worked as a scientific officer for the Govern-ment of India, New Delhi, from 1974 to 1982 andas a faculty member at the University ofWashington, Seattle, from 1985 to 1997 in the

Electrical Engineering and Computer Science and Engineering Depart-ments, where he was promoted to full professor in September 1995.Professor Somani’s research interests are in the area of fault-tolerantcomputing, computer interconnection networks, WDM-based opticalnetworking, and parallel computer system architecture. He is the chiefarchitect of an antisubmarine warfare system (developed for Indiannavy) and Meshkin’s fault-tolerant computer system architecture(developed for the Boeing Company). He has also developed severalrobust interconnection topologies, architected, designed, and imple-mented a 46-node multicomputer cluster-based system, Proteus, usinga large grain message-passing model and separate data and controlplanes, and uses fiber optic communication links. His current research isin developing scalable architectures and algorithms to manage, control,and deliver dependable service efficiently for network employing opticalfiber technology, wavelength division multiplexing, wavelength conver-sion, wavelength sharing, traffic grooming, access network design, andFault and Attack Management (FAM) in optical networking. He hasserved on several program committees of various conferences in hisresearch areas. He was the general chair of IEEE Fault-TolerantComputing Symposium 1997 and technical program committee chair ofthe International Conference on Computer Communications and Net-works 1999 and OPTICOMM 2003. He is serving as the general chair ofBroadNets 2005. He has served as an IEEE distinguished visitor andIEEE distinguished tutorial speaker. He has been elected a fellow of theIEEE for his contributions to theory and applications of computernetworks.

. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.


Documents

CompuP2P: An Architecture for Internet Computing Using Peer-to-Peer Networks