Design and Analysis of Algorithms – Chapter 61 Transform and Conquer (Heaps and Heapsort) Dr. Ying Lu [email protected] RAIK 283: Data Structures & Algorithms

Design and Analysis of Algorithms – Chapter 6 1

Transform and Conquer(Heaps and Heapsort)

Dr. Ying [email protected]

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from

the Textbook’s companion websitethe Textbook’s companion website– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

• Some examples and slides are based on lecture notes Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College and Dr. Ellen and Dr. Chuck Cusack, Hope College and Dr. Ellen Walker from Hiram CollegeWalker from Hiram College

• I have modified many of their slides and added new I have modified many of their slides and added new slides.slides.

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


HeapsortHeapsort

Definition:Definition:A A heapheap is a binary tree with the following conditions: is a binary tree with the following conditions: it is essentially complete: (all levels are full except possibly the last level, it is essentially complete: (all levels are full except possibly the last level,

where only some rightmost leaves may be missing.)where only some rightmost leaves may be missing.)

the key at each node is ≥ keys at its children --- parental the key at each node is ≥ keys at its children --- parental dominancedominance


Heaps (or not)?Heaps (or not)?

7

24

5

9

5

7

26

5

9

1

7

2

5

9

1

tree 1 tree 2

tree 3


Definition implies:Definition implies:

There exists a uniquely structured binary tree with There exists a uniquely structured binary tree with nn nodes nodes that is essentially complete, with that is essentially complete, with hh= lg = lg nn

The root has the largest keyThe root has the largest key

The subtree rooted at any node of a heap is also a heapThe subtree rooted at any node of a heap is also a heap

Partial order tree propertyPartial order tree property


Priority queuesPriority queues

A A priority queue priority queue is the abstract data type (ADT) of is the abstract data type (ADT) of an ordered set with the operations:an ordered set with the operations:• find element with highest priority find element with highest priority

• delete element with highest priority delete element with highest priority

• insert element with assigned priority insert element with assigned priority

Heaps are very good for implementing priority Heaps are very good for implementing priority queuesqueues


Bottom-up vs. Top-down heap Bottom-up vs. Top-down heap constructionconstruction

Bottom-up:Bottom-up: Put everything in and then fix it Put everything in and then fix it

Top down:Top down: Heaps can be constructed by Heaps can be constructed by successively inserting elements into an (initially) successively inserting elements into an (initially) empty heapempty heap


Bottom-up constructionBottom-up construction

Insert elements in the order given breadth-first in Insert elements in the order given breadth-first in a binary treea binary tree

Starting with the last (rightmost) parental node, Starting with the last (rightmost) parental node, fix the heap rooted at it, if it does not satisfy the fix the heap rooted at it, if it does not satisfy the heap condition:heap condition:1.1. exchange it with its largest childexchange it with its largest child2.2. fix the subtree rooted at it (now in the child’s position)fix the subtree rooted at it (now in the child’s position)

Example: 2 9 7 6 5 8 10Example: 2 9 7 6 5 8 10Efficiency:Efficiency:


Bottom-up heap construction Bottom-up heap construction analysis analysis

For parental node at level For parental node at level ii it does 2( it does 2(hh--ii) ) comparisons in the worst casecomparisons in the worst case

Total:Total:

Σ 2(h-i) 2i = 2 ( n – lg (n + 1)) = Θ(n)i=0

h-1

# nodes at level i


Top-down heap constructionTop-down heap construction



Top-down construction: Insertion of a new Top-down construction: Insertion of a new elementelement

Insert element at last position in heapInsert element at last position in heap Compare with its parent and if it violates heap Compare with its parent and if it violates heap

condition exchange themcondition exchange them Continue comparing the new element with nodes Continue comparing the new element with nodes

up the tree until the heap condition is satisfiedup the tree until the heap condition is satisfied

Example: 2 9 7 6 5 8 10Example: 2 9 7 6 5 8 10

Efficiency:Efficiency:


Top-down construction: Insertion of a new Top-down construction: Insertion of a new elementelement

Insert element at last position in heap. Insert element at last position in heap. Compare with its parent and if it violates heap Compare with its parent and if it violates heap

condition exchange themcondition exchange them Continue comparing the new element with nodes Continue comparing the new element with nodes

up the tree until the heap condition is satisfiedup the tree until the heap condition is satisfied

Efficiency: Efficiency: log(i) log(i) n log(n) n log(n)


Bottom-up vs. Top-down heap Bottom-up vs. Top-down heap constructionconstruction

Bottom-up:Bottom-up: Put everything in and then fix it Put everything in and then fix it


Which one is better?Which one is better?


In-class exerciseIn-class exercise

Page 233 Exercise 6.4.1Page 233 Exercise 6.4.1• a. Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by the a. Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by the

bottom-up algorithm. bottom-up algorithm.

• b. Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by b. Construct a heap for the list 1, 8, 6, 5, 3, 7, 4 by successive key insertions (top-down algorithm). successive key insertions (top-down algorithm).

• c. Is it always true that the bottom-up and top-down c. Is it always true that the bottom-up and top-down algorithms yield the same heap for the same input? algorithms yield the same heap for the same input?


Heapsort StrategyHeapsort Strategy

If the elements to be sorted are arranged in a heap, If the elements to be sorted are arranged in a heap, how can we build a sorted sequence from it? how can we build a sorted sequence from it?


Heapsort StrategyHeapsort Strategy

If the elements to be sorted are arranged in a heap, If the elements to be sorted are arranged in a heap, we can build a sorted sequence in reverse order by we can build a sorted sequence in reverse order by • repeatedly removing the element from the root, repeatedly removing the element from the root,

• rearranging the remaining elements to reestablish the rearranging the remaining elements to reestablish the partial order tree property, partial order tree property,

• and so on.and so on.


Heapsort Algorithm:Heapsort Algorithm:

1.1. Build heapBuild heap

2.2. Remove root –exchange with last (rightmost) leafRemove root –exchange with last (rightmost) leaf

3.3. Fix up heap (excluding last leaf)Fix up heap (excluding last leaf)

Repeat 2, 3 until heap contains just one node.Repeat 2, 3 until heap contains just one node.


Root deletion and Heap fix upRoot deletion and Heap fix up

The root of a heap can be deleted and the heap fixed The root of a heap can be deleted and the heap fixed up as follows:up as follows:

exchange the root with the last leaf exchange the root with the last leaf compare the new root (formerly the leaf) with each of its compare the new root (formerly the leaf) with each of its

children and, if one of them is larger than the root, children and, if one of them is larger than the root, exchange it with the larger of the two. exchange it with the larger of the two.

continue the comparison/exchange with its current children continue the comparison/exchange with its current children until it reaches a level of the tree where it is larger than until it reaches a level of the tree where it is larger than

both its childrenboth its children


deleteMax()deleteMax() in actionin action


Analysis of Heapsort Analysis of Heapsort (continued)(continued)

Recall algorithm:Recall algorithm:1.1. Build heapBuild heap






Recall algorithm:Recall algorithm:1.1. Build heapBuild heap




Θ(n)

Θ(log k)

k=n – 1, n-2, … 1

Total:Total: Θ(n) + Θ( n log n) = Θ(n log n)

• Note:Note: this is the worst case. Average case also Θ(n log n).



Is HeapSort an in-place sorting algorithm? Is HeapSort an in-place sorting algorithm?

How to store/represent a Heap in an array?How to store/represent a Heap in an array?


RepresentationRepresentation

Use an array to store breadth-first traversal of heap tree:Use an array to store breadth-first traversal of heap tree: Example:Example:

9

1

5 3

4 2

1 2 3 4 5 6

9 5 3 1 4 2

Left child of node Left child of node jj is at 2 is at 2jj Right child of node Right child of node jj is at 2 is at 2jj+1+1 Parent of node Parent of node jj is at is at jj/2/2 Parental nodes are represented in the first Parental nodes are represented in the first nn/2 locations/2 locations Using array: simpler and efficient implementation of heapsUsing array: simpler and efficient implementation of heaps


Bottom-up heap construction Bottom-up heap construction algorithmalgorithm


Heapsort exampleHeapsort example

Sorting the array 2 9 7 6 5 8 10Sorting the array 2 9 7 6 5 8 10


In-class exercises (I)In-class exercises (I)

Exercise 6.4.7 (a) (c)Exercise 6.4.7 (a) (c)• Sort the following lists by heapsort by using the array Sort the following lists by heapsort by using the array

representation of heaps: representation of heaps: – a) 1, 2, 3, 4, 5 (in increasing order)a) 1, 2, 3, 4, 5 (in increasing order)

– c) S, O, R, T, I, N, G (in alphabetical order)c) S, O, R, T, I, N, G (in alphabetical order)

Design a heap construction algorithm by applying Design a heap construction algorithm by applying divide and conquer strategydivide and conquer strategy

Documents

Design and Analysis of Algorithms – Chapter 61 Transform and Conquer (Heaps and Heapsort) Dr. Ying Lu [email protected] RAIK 283: Data Structures & Algorithms