Amortized Analysis Aggregate Analysis

5/16/2018 Amortized Analysis Aggregate Analysis - slidepdf.com

http://slidepdf.com/reader/full/amortized-analysis-aggregate-analysis-55ab528cd7d01 1/12





LOVELY PROFESSIONAL UNIVERSITY

Algorithm design and analysis

Topic: Amortized analysis: Aggregate method

Submitted to: Submitted by:

Mr. Varun Kohli DEEPAK MISHRA

K1R43b49

10810407



Acknowledgment

First of all I am heartily thankful to my teacher “MR. Varun Kohli” who stand a chance forme to get more knowledge about the topic “Amortized analysis: Aggregate method”

I am also thankful to my friends and my parents who supported me in completion of

this term paper.

And last but not the least very much thankful to the University Providing Internet Facility

inside the campus and providing excellent library facilities hence making the work easier.

DEEPAK MISHRA



Introduction

Amortized analysis is a method of analyzing algorithms that considers the entire sequence

of operations of the program. It allows for the establishment of a worst-case bound for the

performance of an algorithm irrespective of the inputs by looking at all of the operations. At

the heart of the method is the idea that while certain operations may be extremely costly in

resources, they cannot occur at a high-enough frequency to weigh down the entire program

because the number of less costly operations will far outnumber the costly ones in the long

run, "paying back" the program over a number of iterations. It is particularly useful because

it guarantees worst-case performance rather than making assumptions about the state of

the program.

Amortized analysis initially emerged from a method called aggregate analysis, which is now

subsumed by amortized analysis. However, the technique was first formally introduced by

Robert Tarjan in his paper Amortized Computational Complexity, which addressed the need

for a more useful form of analysis than the common probabilistic methods used.

Amortization was initially used for very specific types of algorithms, particularly those

involving binary trees and union operations. However, it is now ubiquitous and comes into

play when analyzing many other algorithms as well.

In an amortized analysis, the time required to perform a sequence of data-structure

operations is averaged over all the operations performed. Amortized analysis can be used to

show that the average cost of an operation is small, if one averages over a sequence of

operations, even though a single operation might be expensive. Amortized analysis differs

from average-case analysis in that probability is not involved; an amortized analysis

guarantees the average performance of each operation in the worst case.

Key ideas

●Amortized analysis is an upper bound: it's the average performance of each operation in th

e worst case.

●Amortized analysis is concerned with the overall cost of a sequence of operations. It does

not say anything about the cost of a specific operation in that sequence. For example, it is i

nvalid toreason, “The amortized cost of insertion into a splay tree with n items is O(log n), so

when I insert '45' into this tree, the cost will be O(log n).” In fact, inserting '45' might re

quire O(n) operations! It is only appropriate to say, “When I insert m items into a tree, t

he average time for each operation will be O(log n).”



●

Amortized analysis is concerned with the overall cost of arbitrary sequences. An amortized

bound will hold regardless of the specific sequence; for example, if the amortized cost of

insertion is O(log n), it is so regardless of whether you insert the sequence '10,' '160,' '2' or t

he sequence '2', '160', '10,' '399', etc.

●The two points above imply that both amortized and worstcase bounds should be

understood when choosing an algorithm to use in practice. Because an amortized bound say

s nothing about the cost of individual operations, it may be possible that one operation in

sequence requires a huge cost. Practical systems in which it is important that all operations

have low and/or comparable costs may require an algorithm with a worse amortized cost

but a better worst-case per operation bound.

●Amortized analysis can be understood to take advantage of the fact that some expensive

operations may “pay” for future operations by somehow limiting the number or cost of

expensive operations that can happen in the near future.

●If good amortized cost is a goal, an algorithm may be designed to explicitly perform this “cl

eanup” during expensive operations

Comparison to other analysis techniques

As mentioned above, worstcase analysis can give overly pessimistic bounds for sequences o

f operations, because such analysis ignores interactions among different operations on the

same data structure (Tarjan 1985). Amortized analysis may lead to a more realistic worstca

se bound by taking these interactions into account. Note that the bound offered by amortized analysis is, in fact, a worstcase bound on the average time per operation; a single operat

ion in a sequence may have cost worse than this bound, but the average cost over all operat

ions in any valid sequence will always perform within the bound.

Amortized analysis is similar to averagecase analysis, in that it is concerned with the cost

averaged over a sequence of operations. However, averagecase analysis relies on probabilis

tic assumptions about the data structures and operations in order to compute an expected

running time of an algorithm. Its applicability is therefore dependent on certain assumptions

about probability distributions on algorithm inputs, which means the analysis is invalid if the

se assumptions do not hold (or that probabilistic analysis cannot be used at all, if input distributions cannot be described!) (Cormenet al. 2001, 92‒3). Amortized analysis needs no such

assumptions. Also, it offers an upperbound on the worst case running time of a sequence of

operations, and this bound will always hold. An averagecase bound, on the other hand, do

es not preclude the possibility that one will get “unlucky” and encounter an input that requi

res much more than the expected computation time, even if the assumptions on the distrib

ution of inputs are valid. These differences between probabilistic and amortized analysis the

refore have important consequences for the interpretation and relevance of the resulting

bounds.

Amortized analysis is closely related to competitive analysis, which involves comparing the



worstcase performance of an online algorithm to the performance of an optimal offline

algorithm on the same data. Amortization is useful because competitive analysis's performa

nce bounds must hold regardless of the particular input, which by definition is seen by the o

nline algorithm in sequence rather than at the beginning of processing. Sleator and Tarjan

(1985a) offer an example of using amortized analysis to perform competitive analysis.

Types:

Amortized analysis is of three types:

1) Aggregate analysis

2) Accounting method

3) Potential method

Aggregate analysis, in which we determine an upper bound T(n) on the total cost of a

sequence of n operations. The amortized cost per operation is then T(n)/n.

Accounting method, in which we determine an amortized cost of each operation. When

there is more than one type of operation, each type of operation may have a different

amortized cost. The accounting method overcharges some operations early in the sequence,

storing the overcharge as "prepaid credit" on specific objects in the data structure. The

credit is used later in the sequence to pay for operations that are charged less than they

actually cost.

The potential method, which is like the accounting method in that we determine the

amortized cost of each operation and may overcharge operations early on to compensate

for undercharges later. The potential method maintains the credit as the "potential energy"

of the data structure instead of associating the credit with individual objects within the data

structure.

We shall use two examples to examine these three models. One is a stack with the

additional operation MULTIPOP, which pops several objects at once. The other is a binary

counter that counts up from 0 by means of the single operation INCREMENT.

The aggregate analysis method

In the aggregate method of amortized analysis, we show that for all n, a sequence of n

operations takes worst-case time T(n) in total. In the worst case, the average cost, or

amortized cost, per operation is therefore T(n) / n. Note that this amortized cost applies to

each operation, even when there are several types of operations in the sequence. The other

two methods we shall study in this chapter, the accounting method and the potential

method, may assign different amortized costs to different types of operations.

Stack operations



In our first example of the aggregate method, we analyze stacks that have been augmented

with a new operation.

PUSH(S, x) pushes object x onto stack S.

POP(S) pops the top of stack S and returns the popped object.

Since each of these operations runs in O(1) time, let us consider the cost of each to be 1.

The total cost of a sequence of n PUSH and POP operations is therefore n, and the actual

running time for n operations is therefore (n).

The situation becomes more interesting if we add the stack operation MULTIPOP(S, k),

which removes the k top objects of stack S, or pops the entire stack if it contains less than k

objects. In the following pseudo code, the operation STACK-EMPTY returns TRUE if there are

no objects currently on the stack, and FALSE otherwise.

MULTIPOP(S,k)

1 while not STACK-EMPTY(S) and k 0

2 do POP(S)

3 k k - 1

Figure 1 shows an example of MULTIPOP.

What is the running time of MULTIPOP(S, k) on a stack of s objects? The actual running time

is linear in the number of POP operations actually executed, and thus it suffices to analyze

MULTIPOP in terms of the abstract costs of 1 each for PUSH and POP. The number of iterations of the while loop is the number min(s, k) of objects popped off the stack. For each

iteration of the loop, one call is made to POP in line 2. Thus, the total cost of MULTIPOP is

min(s, k), and the actual running time is a linear function of this cost.

Figure 1 The action of MULTIPOP on a stack S, shown initially in (a). The top 4 objects are

popped by MULTIPOP(S, 4), whose result is shown in (b). The next operation is MULTIPOP(S,

7), which empties the stack shown in (c) since there were fewer than 7 objects remaining.

Let us analyze a sequence of n PUSH, POP, and MULTIPOP operations on an initially empty

stack. The worst-case cost of a MULTIPOP operation in the sequence is O(n), since the stack

size is at most n. The worst-case time of any stack operation is therefore O(n), and hence a

sequence of n operations costs O(n2), since we may have O(n) MULTIPOP operations costing

O(n) each. Although this analysis is correct, the O(n2

) result, obtained by considering theworst-case cost of each operation individually, is not tight.



Using the aggregate method of amortized analysis, we can obtain a better upper bound that

considers the entire sequence of n operations. In fact, although a single MULTIPOP

operation can be expensive, any sequence of n PUSH, POP, and MULTIPOP operations on an

initially empty stack can cost at most O(n). Why? Each object can be popped at most once

for each time it is pushed. Therefore, the number of times that POP can be called on a

nonempty stack, including calls within MULTIPOP, is at most the number of PUSH

operations, which is at most n. For any value of n, any sequence of n PUSH, POP, and

MULTIPOP operations takes a total of O(n) time. The amortized cost of an operation is the

average: O(n)/n = O(1).

We emphasize again that although we have just shown that the average cost, and hence

running time, of a stack operation is O(1), no probabilistic reasoning was involved. We

actually showed a worst-case bound of O(n) on a sequence of n operations. Dividing this

total cost by n yielded the average cost per operation, or the amortized cost.

Incrementing a binary counter

As another example of the aggregate method, consider the problem of implementing a k-bit

binary counter that counts upward from 0. We use an array A[0 . . k - 1] of bits, where

length[A] = k, as the counter. A binary number x that is stored in the counter has its lowest-

order bit in A[0] and its highest-order bit in A[k - 1], so that . Initially, x = 0,

and thus A[i] = 0 for i = 0, 1, . . . , k - 1. To add 1 (modulo 2k) to the value in the counter, we

use the following procedure.

Amortized cost per operation is T(n)/nX A[4] A[3] A[2] A[1] A[0] Total Cost

0 0 0 0 0 0 0

1 0 0 0 0 1 1

2 0 0 0 1 0 3

3 0 0 0 1 1 4

4 0 0 1 0 0 7

5 0 0 1 0 1 8

6 0 0 1 1 0 10

7 0 0 1 1 1 11

8 0 1 0 0 0 15

9 0 1 0 0 1 16



Figure 2 An 8-bit binary counter as its value goes from 0 to 16 by a sequence of 16

INCREMENT operations. Bits that flip to achieve the next value are shaded. The running cost

for flipping bits is shown at the right. Notice that the total cost is never more than twice the

total number of INCREMENT operations.

INCREMENT (A)

1 i 0

2 while i< length[A] and A[i] = 1

3 do A[i] 0

4 i i + 1

5 if i < length[A]

6 then A[i] 1

This algorithm is essentially the same one implemented in hardware by a ripple-carry

counter Figure 2 shows what happens to a binary counter as it is incremented 16 times,

starting with the initial value 0 and ending with the value 16. At the start of each iteration of the while loops in lines 2-4, we wish to add a 1 into position i. If A[i] = 1, then adding 1 flips

the bit to 0 in position i and yields a carry of 1, to be added into position i + 1 on the next

iteration of the loop. Otherwise, the loop ends, and then, if i < k, we know that A[i] = 0, so

that adding a 1 into position i, flipping the 0 to a 1, is taken care of in line 6. The cost of each

INCREMENT operation is linear in the number of bits flipped.

As with the stack example, a cursory analysis yields a bound that is correct but not tight. A

single execution of INCREMENT takes time (k) in the worst case, in which array A contains

all 1's. Thus, a sequence of n INCREMENT operations on an initially zero counter takes time

O(nk) in the worst case.

We can tighten our analysis to yield a worst-case cost of O(n) for a sequence of n

INCREMENT'S by observing that not all bits flip each time INCREMENT is called. As Figure 2

shows, A[0] does flip each time INCREMENT is called. The next-highest-order bit, A[1], flips

only every other time: a sequence of n INCREMENT operations on an initially zero counter

causes A[1] to flip n/2 times. Similarly, bit A[2] flips only every fourth time, or n/4 times

in a sequence of n INCREMENT'S. In general, for i = 0, 1, . . . , lg n , bit A[i] flips n/2i

times

in a sequence of n INCREMENT operations on an initially zero counter. For i > lg n , bit A[i]

never flips at all. The total number of flips in the sequence is thus

by equation (3.4). The worst-case time for a sequence of n INCREMENT operations on an

initially zero counter is therefore O(n), so the amortized cost of each operation is O(n)/n =

O(1).



Conclusion

Amortized analysis is a useful tool that complements other techniques such as worstcase a

nd averagecase analysis. It has been applied to a variety of problems, and it is also crucial

appreciating structures such as splay trees that have been designed to have good amortized

bounds.

To understand the application of amortized analysis to common problems, it is essential to

know the basics of both the accounting method and the potential method. The resources

presented here supply many examples of both methods applied to real problems.

To perform an amortized analysis, one should choose either the accounting method or the

potential method. The approaches yield equivalent results, but one might be more intuitivey

appropriate to the problem under consideration. There is no magic formula for arriving at a

potential function or accounting credit scheme that will always work; the method used depe

nds on the desired bounds and the desired complexity of the analysis. Some strategies that

sometimes work, however, include enumerating the ways in which an algorithm might

operate on a data structure, then performing an analysis for each case; computing the pote

ntial of a data structure as a sum of “local” potentials so that one can reason about the effec

ts of local changes while ignoring irrelevant and unchanging components of the structure; d

esigning the potential method around the desired form of the result (e.g., relating the poten

tial to the log of the subtree size for splay trees, as the desired outcome is a logarithmic bou

nd); and reasoning about each type of operation in a sequence individually before coming u

p with a bound on an arbitrary sequence of operations.

An understanding of amortized analysis is essential to success in an algorithms course, to

understanding the implication of theoretical bounds on realworld performance, and to

thoroughly appreciating the design and purpose of certain data structures. The reader is the

refore again urged to consult any of the sources mentioned here to improve his or her

understanding of amortized analysis and to explore these algorithms in greater depth.



References:-

1) http://www.eli.sdsu.edu/courses/fall95/cs660/notes/amortized/Amortized.html#

RTFToC3

2) http://staff.ustc.edu.cn/~csli/graduate/algorithms/book6/chap18.htm

3) http://www.bowdoin.edu/~ltoma/teaching/cs231/fall11/Lectures/13-

amortized/amortized.pdf

Documents

Amortized Analysis Aggregate Analysis