Algorithms and Data Structures - BFHhnr1/algdata/slides/05Heaps.pdf · Priority Queues and Heaps Page 1 Berner Fachhochschule - Technik und Informatik Algorithms and Data Structures

Priority Queues and Heaps Page 1

Berner Fachhochschule - Technik und Informatik

Algorithms and Data Structures

Priority Queues and Heaps

Dr. Rolf Haenni

Fall 2008

Berner Fachhochschule Rolf Haenni

Technik und Informatik Algorithms and Data Structures

Priority Queues and Heaps Page 2

Outline

Priority Queues

Heaps

Heap-Based Priority Queues

Bottom-Up Heap Construction



Priority Queues and Heaps Page 3Priority Queues

Outline

Priority Queues

Heaps






Priority Queue ADT

I A priority queue stores a collection of items

I An item is a pair (key , element)I Characteristic operations:

Ý insertItem(k,e): inserts an item with key k and element eÝ removeMin(): removes the item with the smallest key and

returns its elementÝ minKey(): returns the smallest key of an item (no removal)Ý minElement(): returns the element of an item with smallest

key (no removal)

I General operations:

Ý size(): returns the number of itemsÝ isEmpty(): indicates whether the priority queue is empty

I Two distinct items in a priority queue can have the same key




Total Order

I Keys in a priority queue can be arbitrary objects on which atotal order is defined

I Mathematically, a total order is a binary relation � defined ona set K which satisfies three properties:

Ý Totality: x � y or y � x , for all x , y ∈ KÝ Antisymmetry: x � y and y � x implies x = y , for all x , y ∈ KÝ Transitivity: x � y and y � z implies x � z , for all x , y , z ∈ K

I Totality implies Reflexivity: x � x , for all x ∈ K

I Examples: ≤ or ≥ for R, alphabetical or reverse alphabeticalorder for {A, . . . ,Z}, lexicographical order for {A, . . . ,Z}∗,etc.




Comparator ADT

I A comparator encapsulates the action of comparing two keysaccording to a given total order relation

I A generic priority queue uses an auxiliary comparator (passedas a parameter to the constructor)

I When the priority queue needs to compare two keys, it uses itscomparator

I Operations (all with Boolean return type):

Ý isLessThan(x,y)Ý isLessThanOrEqualTo(x,y)Ý isEqualTo(x,y)Ý isGreaterThan(x,y)Ý isGreaterThanOrEqualTo(x,y)Ý isComparable(x)




Sequence-Based Priority Queue

I There are two ways to implement a priority queue with asequence (list or vector)

I Using an unsorted sequence

Ý insertItem(k,e) runs in O(1) time, since we can insert theitem at the beginning of the sequence

Ý removeMin(), minKey(), minElement() run in O(n) timesince we have to traverse the entire sequence to find thesmallest key

I Using a sorted sequence

Ý insertItem(k,e) runs in O(n) time, since we have to findthe place where to insert the item

Ý removeMin(), minKey(), minElement() run in O(1) timesince the smallest key is at the beginning or end of thesequence




UML Diagram: Composition

<<interface>>PriorityQueue

insertItem(k,e)removeMin()minKey()minElement()

<<interface>>Position

etc.

<<interface>>Comparator

isLessThan(x,y)isEqualTo(x,y)etc.

<<interface>>List

etc.

<<interface>>BasicSequence

isEmpty()size()

SinglyLinkedListfirst: ListNoden: Integer

ListNodeelement: Objectnext: ListNode

SortedPQlist: SinglyLinkedListC: Comparator

UnsortedPQlist: SinglyLinkedListC: Comparator




UML Diagram: Inheritance



<<interface>>Position

etc.



<<interface>>List

etc.


isEmpty()size()

SinglyLinkedListfirst: ListNoden: Integer

ListNodeelement: Objectnext: ListNode

SortedPQC: Comparator

UnsortedPQC: Comparator




Sorting with a Priority Queue

I We can use a priority queue to sort a collection of comparableelements

Ý Insert the elements one by one with a series ofinsertItem(e,e) operations

Ý Remove the elements in sorted order with a series ofremoveMin() operations

I Depending on how the priority queue is implemented (sortedor unsorted), this leads to two well-known sorting algorithms

Ý Selection-SortÝ Insertion-Sort




PQ-Sort Algorithm

Solution in pseudo-code:

Algorithm PQ−Sort(S ,C )P ← new priority queue with comparator Cwhile not S .isEmpty() do

e ← S .removeElement(S .first())P.insertItem(e,e)

while not P.isEmpty() doe ← P.removeMin()S .insertLast(e)

return S




Selection-Sort

I Selection-sort is the variation of PQ-sort where the priorityqueue is implemented with an unsorted sequence

I The main task is to select the elements from the unsortedsequence in the right order

I Running time analysis:

Ý Inserting the elements into the priority queue with ninsertItem(e,e) operations runs in O(n) time

Ý Removing the elements in sorted order from the priority queuewith n removeMin() operations takes time proportional ton + . . . + 2 + 1, and thus runs in O(n2) time

I Selection-sort runs in O(n2) time




Insertion-Sort

I Selection-sort is the variation of PQ-sort where the priorityqueue is implemented with an sorted sequence

I The main task is to insert the elements at the right placeI Running time analysis:

Ý Inserting the elements into the priority queue with ninsertItem(e,e) operations takes time proportional to1 + 2 + . . . + n, and thus runs in O(n2) time

Ý Removing the elements in sorted order from the priority queuewith n removeMin() operations runs in O(n) time

I In general (worst case), insertion-sort runs in O(n2) time

I If the input sequence is in reverse order (best case), it runs inO(n) time




In-Place Sorting

I Instead of using external datastructures, we can implementinsertion- and selection-sortin-place

I A portion of the input sequenceitself serves as priority queue

I For in-place insertion-sort we

Ý keep the initial portion of thesequence sorted

Ý use swapElements(p,q) insteadof modifying the sequence

4 5 2 3 1

2 4 5 3 1

2 3 4 5 1

1 2 3 4 5

5 4 2 3 1

5 4 2 3 1



Priority Queues and Heaps Page 15Heaps

Outline

Priority Queues

Heaps






The Heap ADT

I A heap is a specialized binary tree storing keys at its nodes(positions) and satisfying the following properties:

Ý Heap-Order: key(p) � key(parent(p)), for every node p otherthan the root

Ý Completeness: let h be the height of the tree1) there are 2i nodes of depth i for i = 0, . . . , h − 12) at depth h, nodes are filled up from the left

I The last node of a heap is the rightmost node of depth hI Heap operations:

Ý insertKey(k): inserts a key k into the heapÝ removeMin(): removes smallest key and returns itÝ minKey(): returns the smallest key (no removal)




Heap Example

52

6

9 7 11 8

13 10 9 8

Last node

01

2

312 14 9 11

17 15 412

Depth

I A heap storing n keys has height O(log n)




Insertion to a Heap

I The insertion of a key kconsists of 3 steps:

Ý Add a new node q (the newlast node)

Ý Store k at qÝ Restore the heap order

property (discussed next)

52

6

9 7

52

6

9 7 1

q




Upheap

I Algorithm upheap restores theheap-order property

Ý Swap k along an upward pathfrom the insertion node

Ý Stop when k reaches the rootor a node whose parent has akey smaller than or equal to k

Ý Since the height of the heapis O(log n), upheap runs inO(log n) time

I insertKey(k) runs in O(log n)time

52

1

9 7

51

2

9 7 6

6




Removal from a Heap

I The removal algorithm consistsof 4 steps:

Ý Return the key of the rootÝ Replace the root key with the

key k of the last nodeÝ Delete the last nodeÝ Restore the heap order

property (discussed next)

52

6

9 7

57

6

9

Last node




Downheap

I Algorithm downheap restores theheap-order property

Ý Swap k along an downwardpath from the root (always withthe smaller key of its children)

Ý Stop when k reaches a leaf or anode whose children have keysgreater than or equal to k

Ý Since the height of the heap isO(log n), downheap runs inO(log n) time

I removeMin() runs in O(log n)time

57

6

9

75

6

9




Finding the Insertion Node

I Starting from the last node, the insertion node can be foundby traversing a path of O(log n) nodes

Ý If the last node is a left child, return the parent nodeÝ While the current node is a right child, go to the parent nodeÝ If the current node is a left child, go to the right childÝ While the current node is internal, go to the left child

52

6

9 7 11 8

13 10 9 8 Last node Insertion node




Array-Based Heap Implementation

I We can represent a heap with n keys directly by means of anarray of length N > n

Ý By-pass the binary tree ADTÝ No explicit position ADT needed

I Idea similar to array-based implementation of binary trees

Ý The array cell at index 0 is unusedÝ The root of the heap is at index 1Ý The left child of the node at index i is at index 2iÝ The right child of the node at index i is at index 2i + 1

I The insertion operations corresponds to inserting at indexn + 1 and thus runs in O(1) time




Array-Based Heap: Example

52

6

9 7 11 8

13 10 9 8

52 9 7 11 8 13 10 9 861 2 3 4 5 6 7 8 9 10 11 12 13 140

Last node



Priority Queues and Heaps Page 25Heap-Based Priority Queues

Outline

Priority Queues

Heaps







I We can use a heap to implement a priority queue by storing apair item = (key , element) at each node of the heap

I Running times for different implementations

Unsorted SortedOperation Sequence Sequence Heapsize, isEmpty 1 1 1minElement, minKey n 1 1insertItem 1 n log nremoveMin n 1 log nPQ-Sort n2 n2 n· log n

I In the long run, the heap-based implementation beats anysequence-based implementation




UML Diagram: Composition

<<interface>>BinaryTree

etc.

<<interface>>Tree

etc.

ArrayBinaryTreeT: Arrayn: Integer

<<interface>>Heap

insertKey(k)removeMin()minKey()

BinaryTreeHeapT: ArrayBinaryTreeC: Comparator




isEmpty()size()

HeapPQH: ArrayHeapC: Comparator



ArrayHeapA: ArrayC: Comparator




Heap-Sort

I Heap-sort is the variation of PQ-sort where the priority queueis implemented with a heap

I Running time analysis:

Ý Inserting the elements into the priority queue with ninsertItem(e,e) operations runs in O(n log n) time

Ý Removing the elements in sorted order from the priority queuewith n removeMin() operations runs in O(n log n) time

I In general (worst case), heap-sort runs in O(n log n) time

I For large n, heap-sort is much faster than quadratic sortingalgorithms such as insertion-sort or selection-sort




In-Place Heap-Sort

I To implement Heap-Sort in-place, use the front of the originalarray to implement the heap

Ý Step 1: Use the reverse comparator (e.g. � instead of �) tobuild up the heap (by swapping elements)

Ý Step 2: Iteratively remove the maximum element from theheap and insert it in front of the sequence

4 7 2 5 3 4 7 2 5 3 7 4 2 5 3 7 4 2 5 3 7 5 2 4 3 7 5 2 4 34 7 7

4 4 27

5 24

75 2

4 3

7 5 2 4 3

75 2

4 35 4 2 3 7

54 2

34 3 2 5 7

43 2

3 2 4 5 7

32

2 3 4 5 72

2 3 4 5 7

Phase 1:

Phase 2:



Priority Queues and Heaps Page 30Bottom-Up Heap Construction

Outline

Priority Queues

Heaps






Bottom-up Heap Construction

I We can construct a heap storing n = 2h − 1 keys using arecursive bottom-up construction with h − 1 phases

I In phase i , pairs of heaps with 2i − 1 keys are merged intoheaps with 2i+1 − 1 keys

I Merging two heaps (and a new key k):

Ý Create a new heap with the root node storing k and with thetwo heaps as subtrees

Ý Perform downheap to restore the heap-order property

2i–1 2i–12i+1–1




Recursive Algorithm in Pseudo-Code

Algorithm bottomUpHeap(S) // sequence of keys of length 2ˆh−1if S . isEmpty() thenreturn new heap

elsek ← S . elemAtRank(0)S . removeAtRank(0)H1 ←bottomUpHeap(firstHalf(S)) // 1st recursive callH2 ←bottomUpHeap(secondHalf(S)) // 2nd recursive callreturn mergeHeaps(k,H1,H2) // merge the results

In the general case, i.e. if the sequence is not of size n = 2h − 1,we can adapt the splitting of the sequence accordingly




Example: Phase 1

16 15 4 12 6 9 23 20

25 5 11 27

16 15 4 12 6 9 23 20

15 4 6 20

16 25 5 12 11 9 23 27




Example: Phase 2

7 8

15 4 6 20

16 25 5 12 11 9 23 27

15 4 6 20

16 25 5 12 11 9 23 27

4 6

15 5 8 20

16 25 7 12 11 9 23 27




Example: Phase 3

4 6

15 5 8 20

16 25 7 12 11 9 23 27

410

6

15 5 8 20

16 25 7 12 11 9 23 27

54

6

15 7 8 20

16 25 10 12 11 9 23 27




Complexity Analysis

I The recursive BottomUpHeap algorithm decomposes aproblem of size n into two problems of size n−1

2

Ý Rule D2 with q = r = 2 (see Topic 2, Page 31)

I The running time f (n) of each recursive step is O(log n)

Ý with an array-based sequence implementation, firstHalf()and secondHalf() run in O(1) time

Ý the necessary downheap in mergeHeaps runs in O(log n) time

I Apply the second column of D2, i.e. bottom-up heapconstruction runs in O(n)

I Speeds up the heap construction (Phase 1) of the Heap-Sortalgorithm, but not Phase 2



Documents

Algorithms and Data Structures - BFHhnr1/algdata/slides/05Heaps.pdf · Priority Queues and Heaps Page 1 Berner Fachhochschule - Technik und Informatik Algorithms and Data Structures