heaps.pdf

8/9/2019 heaps.pdf

1/72

111111Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

2/72

Motivation

Queues are a standard mechanism for ordering taskson a first-come, first-served basis

However, some tasks may be more important ortimely than others (higher priority)

Store tasks using a partial ordering based on priority

Ensure highest priority task at head of queue

Heaps are the underlying data structure of priorityqueues


8/9/2019 heaps.pdf

3/72

Priority Queues: Specification

Main operations insert (i.e., enqueue)

ynam c nser

specification of a priority level (0-high, 1,2.. Low)

deleteMin (i.e., dequeue)

Finds the current minimum element (read: “highest priority”) inthe queue, deletes it from the queue, and returns it

Performance goal is for operations to be “fast”


8/9/2019 heaps.pdf

4/72

Using priority queues

5 3

1019

4

8

2211

insert()

deleteMin()

2Dequeues the next element


8/9/2019 heaps.pdf

5/72

Can we build a data structure better suited to store and retrieve priorities?

Simple Implementations

Unordered linked list O(1) insert

5 2 103…

Ordered linked list O(n) insert

2 3 5 10…

e e e n

Ordered array O(lg n + n) insert

2 3 5 … 10

O(n) deleteMin

Balanced BST


8/9/2019 heaps.pdf

6/72

nary eap

A priority queue data structure


8/9/2019 heaps.pdf

7/72

Binary Heap

A binary heap is a binary tree with two

Structure property

-


8/9/2019 heaps.pdf

8/72

Structure Property

A binary heap is a complete binary tree Each level exce t ossibl the bottom most level

is completely filled

The bottom most level may be partially filledrom e o r g

Height of a complete binary tree with Nelements is N 2log


8/9/2019 heaps.pdf

9/72

Structure property

Binary Heap Example

N=10

Every level

(except last)

saturated

Array representation:


8/9/2019 heaps.pdf

10/72

Heap-order Property

Heap-order property (for a “MinHeap”) ,

Except root node, which has no parent

,

Alternatively, for a “MaxHeap”, always

Insert and deleteMin must maintain-


8/9/2019 heaps.pdf

11/72

Heap Order Property

Minimum

element

No order implied for elements which do not

-


8/9/2019 heaps.pdf

12/72

Implementing CompleteBinary Trees as Arrays

Given element at position i in the array=

Right child(i) = at position 2i + 1

2i

2i + 1

i


8/9/2019 heaps.pdf

13/72

insert

without deleting it

deleteMin

Note: a general delete()function is not as important

for heaps

but could be implemented

Stores the heap as

a vector

Fix hea after

13

deleteMin

Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

14/72

Heap Insert

Insert new element into the heap at the “ ”

According to maintaining a complete binary

tree

Then, “percolate” the element up the

- satisfied


8/9/2019 heaps.pdf

15/72

Percolating Up

Heap Insert: Example

Insert 14:

hole14


8/9/2019 heaps.pdf

16/72

Percolating Up


Insert 14:(1)

14 vs. 31

hole

14

14


8/9/2019 heaps.pdf

17/72

Percolating Up


Insert 14:(1)

14 vs. 31

hole

14

14

14 vs. 21

14


8/9/2019 heaps.pdf

18/72

Percolating Up


Insert 14:(1)

14 vs. 31

hole14

(3) Heap order prop14

14 vs. 21

vs. ruc ure prop

Path of ercolation u


8/9/2019 heaps.pdf

19/72

Heap Insert: Implementation

// assume array implementationvoid insert( const Comparable &x) {

}


8/9/2019 heaps.pdf

20/72

Heap Insert: Implementation


8/9/2019 heaps.pdf

21/72

Heap DeleteMin

Minimum element is always at the root

Move last element into hole at root Percolate down while heap-order

property not satisfied


8/9/2019 heaps.pdf

22/72

Percolating down…

Heap DeleteMin: Example

Make thisposition

empty


8/9/2019 heaps.pdf

23/72

Percolating down…


Copy 31 temporarilyhere and move it dow

Is 31 > min(14,16)?

•Yes - swap 31 with min(14,16)

Make thisposition

empty


8/9/2019 heaps.pdf

24/72

Percolating down…


31

Is 31 > min(19,21)?



8/9/2019 heaps.pdf

25/72

Percolating down…


31

31

Is 31 > min(65,26)?

•Yes - swap 31 with min(65,26)Is 31 > min(19,21)?


25Percolating down… 25Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

26/72

Percolating down…


31

26Percolating down…Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

27/72

Percolating down…


31

Heap order prop

27


8/9/2019 heaps.pdf

28/72

Heap DeleteMin:Implementation

28

O(log N) time


8/9/2019 heaps.pdf

29/72

Heap DeleteMin:Implementation

Percolate

Left child

down

Right child

Pick child toswap with


8/9/2019 heaps.pdf

30/72

Other Heap Operations

decreaseKey(p,v) Lowers the current value of item p to new priority value v Need to ercolate u E.g., promote a job

increaseKey(p,v) Increases the current value of item to new riorit value v Need to percolate down E.g., demote a job

remove Run-times for all three functions? First, decreaseKey(p,-∞) Then, deleteMin E.g., abort/cancel a job

O(lg n)


8/9/2019 heaps.pdf

31/72

Improving Heap Insert Time

What if all N elements are all availableupfront?

o build a hea with N elements: Default method takes O(N lg N) time

We will now see a new method called buildHeap()

t at wi ta e O N time - i.e., optima


8/9/2019 heaps.pdf

32/72

Building a Heap

Construct heap from initial set of N items Solution 1

Perform N inserts

O(N log2 N) worst-case Solution 2 (use buildHeap() )

Randomly populate initial heap with structure

property Perform a percolate-down from each internal node

To take care of heap order property32Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

33/72

BuildHeap Examplenput: , , , , , , , , , , , , , ,

valid heaps

(implicitly)

• Arbitrarily assign elements to heap nodes

• Structure property satisfied

• Hea order ro ert violated

So, let us look at eachinternal node,

from bottom to top,

33

• Leaves are all valid heaps (implicit)


8/9/2019 heaps.pdf

34/72

BuildHeap ExampleSwap

Nothingto do

with left

child

• Randomly initialized heap

34

• Structure property satisfied

• Heap order property violated

• Leaves are all valid heaps (implicit) 34Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

35/72

BuildHeap Example Swapwith right

childNothing

to do

Dotted lines show path of percolating down


8/9/2019 heaps.pdf

36/72

Swa with

BuildHeap Exampleright child

& then with 60

o ng

to do



8/9/2019 heaps.pdf

37/72

BuildHeap Example

Swap path


Final Heap


8/9/2019 heaps.pdf

38/72

BuildHeap Implementation

Start withlowest,

rightmost

38

internal node


8/9/2019 heaps.pdf

39/72

BuildHeap() : Run-time Analysis

Run-time = ? O(sum of the heights of all the internal nodes)

ecause we may ave o perco a e a e waydown to fix every internal node in the worst-case

heorem 6.1 HOW? For a perfect binary tree of height h, the sum of

heights of all nodes is 2 h+1 – 1 – (h + 1)

nce = g , en sum o e g s s Will be slightly better in practice

Implication: Each insertion costs O(1) amortized time


8/9/2019 heaps.pdf

40/72


8/9/2019 heaps.pdf

41/72

Binary Heap OperationsWorst-case Analysis

Height of heap is insert: O l N for each insert

N 2log

In practice, expect less

buildHea insert: O N for N inserts deleteMin: O(lg N)

r K : l N

increaseKey: O(lg N)


8/9/2019 heaps.pdf

42/72

Applications

Operating system scheduling

Graph algorithms

Event simulation

Instead of checking for events at each timeclick, look up next event to happen


8/9/2019 heaps.pdf

43/72

An Application:he Selection Problem

Given a list of n elements, find the k th

smallest element

Sort the list => O(n log n)

Pick the k th element => O 1

A better algorithm:


8/9/2019 heaps.pdf

44/72

Selection using a MinHeap

Input: n elements

Algorithm:1. u eap n == n

2. Perform k deleteMin() operations ==> O(k log n)

3.

Report the k th

deleteMin output ==> O(1)

Total run-time = O(n + k log n)

If k = O(n/log n) then the run-time becomes O(n)


8/9/2019 heaps.pdf

45/72

Other Types of Heaps

Binomial Heaps

- eaps Generalization of binary heaps (ie., 2-Heaps)

Leftist Heaps Supports merging of two heaps in o(m+n) time (ie., sub-

linear)

Skew Heaps O(log n) amortized run-time

Fibonacci Heaps45Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

46/72

Run-time Per Operation

Insert DeleteMin Merge (=H1+H2)

Binary heap O(log n) worst-case O(log n) O(n)

O(1) amortized forbuildHeap

Leftist Heap O(log n) O(log n) O(log n)

Skew Heap O(log n) O(log n) O(log n)

nom aHeap

og n wors case

O(1) amortized forsequence of n inserts

og n og n

Fibonacci Heap O(1) O(log n) O(1)


8/9/2019 heaps.pdf

47/72

Priority Queues in STL

Uses Binary heap Default is MaxHea

#include

int main ()

Methods

Push to o

{

priority_queue Q;

Q.push (10);

empty, clear .

Q.pop ();

}Calls DeleteMax()

For MinHeap: declare priority_queue as:

priority_queue Q;

47

Refer to Book Chapter 6, Fig 6.57 for an example


8/9/2019 heaps.pdf

48/72

Binomial Heaps


8/9/2019 heaps.pdf

49/72

Binomial Heap

A binomial heap is a forest of heap-orderedbinomial trees, satisfying:

,ii) Heap order property

A binomial heap is different from binary heapin that:

Its heap-order property (within each binomial

tree) is the same as in a binary heap


Note: A binomial tree need not be a binary tree

8/9/2019 heaps.pdf

50/72

y

Definition: A “Binomial Tree” Bk

A binomial tree of height k is called Bk : It has 2k nodes

The number of nodes at depth d = ( )k

d

-k d

3

Depth: #nodes:B3:

=

d=1

0

( 31 )3

d=3

2

( 33 ) 50Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

51/72

Wh will Bin mi l H wi h n= 1 nodes look like?

We know that:i) A binomial heap should be a forest of binomial

trees

ii) Each binomial tree has power of 2 elements

B0B1B2B3B4

o ow many inomia trees o we nee

n = =2


8/9/2019 heaps.pdf

52/72

nom a eap w n= no es

n = 31 = (1 1 1 1 1)2

Bi == Bi-1 + Bi-1

, B 2 ,

B 3 ,

B 4

}

BB

B1B0

t r e e s { B 0 ,

B 1

t o f b i n o m

i a l

F o r e s


8/9/2019 heaps.pdf

53/72

Binomial Heap Property

Lemma: There exists a binomial heap for everypositive value of n

Proof:

Have one binomial tree for each power of two with co-efficientof 1

= == ==


Binomial Heaps: Heap Order

8/9/2019 heaps.pdf

54/72

Binomial Heaps: Heap-OrderProperty

Each binomial tree should contain theminimum element at the root of everysubtree

Just like binary heap, except that the treeere s a nom a tree structure an not acomplete binary tree)

The order of elements across binomial


8/9/2019 heaps.pdf

55/72

Definition: Binomial Heaps

A binomial heap of n nodes is:

(Structure Property) A forest of binomial trees as dictated by

(Heap-Order Property) Each binomial tree is a min-heap or amax- eap


8/9/2019 heaps.pdf

56/72

Binomial Heaps: Examples

Two different heaps:


8/9/2019 heaps.pdf

57/72

Key Properties

Could there be multiple trees of the same height in abinomial heap?

no

What is the upper bound on the number of binomial

g n

Given n , can we tell (for sure) if Bk exists?

Bk exists if and only if:

the kth least significant bit is 1

in the binary representation of n


8/9/2019 heaps.pdf

58/72

An Im lementation of a Binomial HeaMaintain a linked list of

tree pointers (for the forest)

B0

B1

B2

B3

B4

B5

B6

B7

2

Shown using the

Analo ous to a bit-based re resentation of a

- , -

binary number n


8/9/2019 heaps.pdf

59/72

Binomial Heap: Operations

x

8/9/2019 heaps.pdf

60/72

DeleteMin()

Goal: Given a binomial heap, H, find theminimum and delete it

Observation: The root of each binomial tree

in H contains its minimum element Approach: Therefore, return the minimum of

all the roots (minimums)

Complexity: O(log n) comparisons(because there are only O(log n) trees)


8/9/2019 heaps.pdf

61/72

FindMin() & DeleteMin() Example

B0 B2B3

B1’ B2’B0’

For DeleteMin(): After delete, how to adjust the heap?

’ ’ ’ 0, 2 0 , 1 , 2


8/9/2019 heaps.pdf

62/72

Insert(x ) in Binomial Heap

Goal: To insert a new element x into a

Observation:

element binomial heap

= == ,

So, if we decide how to do merge we will automatically

figure out how to implement both insert() and deleteMin()


8/9/2019 heaps.pdf

63/72

Merge(H1,H2)

Let n1 be the number of nodes in H1 Let n2 be the number of nodes in H2

, 1 2nodes Assume n = n1 + n2

Logic: Merge trees of same height, starting from lowest height

If only one tree of a given height, then just copy that Otherwise, need to do carryover (just like adding two binary

numbers)


Idea: merge tree of same heights

8/9/2019 heaps.pdf

64/72

Merge: Example

+

B0 B1 B2

13

?? 64Cpt S 223. School of EECS, WSU

How to Merge Two Binomial

8/9/2019 heaps.pdf

65/72

How to Merge Two Binomial

rees of the Same Height?

+

B2:

B2: B3:

Simply compare the roots

Note: Merge is defined for only binomial trees with the same height 65Cpt S 223. School of EECS, WSU

8/9/2019 heaps.pdf

66/72

1, 2 carryover

+

13 14 ?

26

26


How to Mer e more than two

8/9/2019 heaps.pdf

67/72

How to Mer e more than two binomial trees of the same height?

Merging more than 2 binomial trees ofthe same height could generate carry-overs

+14

26 16

26

-


Input:

8/9/2019 heaps.pdf

68/72

+

Merge(H1,H2) : Example

Output:

w w

= ,


8/9/2019 heaps.pdf

69/72

Run-time Complexities

Merge takes O(log n) comparisons

Insert and DeleteMin also take O(log n)

Insert operations takes only O(m) time per operation, implyingO(1) amortize time per insert

Proof Hint: For each insertion, if i is the least significant bit position with a 0, then

number of comparisons required to do the next insert is i+1 If you count the #bit flips for each insert, going from insert of the first

element to the insert of the last (nth) element, thenaffected

unaffected

= amor ze run- me o per nser

1

--------------

10011000 69Cpt S 223. School of EECS, WSU

Binomial Queue Run-time

8/9/2019 heaps.pdf

70/72

Q

Summary

Insert O(lg n) worst-case

O(1) amortized time if insertion is done in anuninterrupted sequence (i.e., without being

intervened by deleteMins) DeleteMin, FindMin

O(lg n) worst-case

Merge O(lg n) worst-case


8/9/2019 heaps.pdf

71/72

Run-time Per Operation

Insert DeleteMin Merge (=H1+H2)

Binary heap O(log n) worst-case O(log n) O(n)

O(1) amortized forbuildHeap

Leftist Heap O(log n) O(log n) O(log n)

Skew Heap O(log n) O(log n) O(log n)

nom a

Heap

og n wors case

O(1) amortized forsequence of n inserts

og n og n

Fibonacci Heap O(1) O(log n) O(1)


8/9/2019 heaps.pdf

72/72

Summary

Priority queues maintain the minimum

Support O(log N) operations worst-case

, ,

Many applications in support of othera or ms


Documents

heaps.pdf