Data Structures Hanoch Levi and Uri Zwick March 2011 Lecture 3 Dynamic Sets / Dictionaries Binary Search Trees

Data Structures

Hanoch Levi and Uri ZwickMarch 2011

Lecture 3Dynamic Sets / Dictionaries

Binary Search Trees

2

Dictionaries/Dynamic sets• Maintain a set of items.• Each item has a key associated with it.• Keys belong to a totally ordered universe,

and can be compared with each other• Support the following operations:

Insert, Delete, Search, Min, Max, …

Extremely useful data structure!

Abstract Data Type: DictionariesDic-Item(i,k) – Create a dictionary item containing object i with key kItem(a), Key(a) – The item and key contained in Dec-Item a.

Dictionary() – Create an empty dictionaryInsert(D,a) – Insert a into DDelete(D,a) – Delete a from D (assuming a is in D)Search(D,k) – Find a dic-item with key k in D, if any.Min(D) – Return the dic-item with the minimum key in D.Max(D) – Return the dic-item with the maximum key in D.Successor(D,a) – Return the successor of a in D. Predecessor(D,a) – Return the predecessor of a in D.

Assume that dic-items have distinct keys

4

Implementing dictionaries using lists

• Store the dic-items in a list (in no particular order).• Insert a new dic-item to an arbitrary position of the list,

e.g., the first or last position.• Delete a dic-item by either using a supplied pointer to it,

or by first locating it in the list.• Search, and other operations, are implemented by scanning

the list.

5

Implementing dictionaries using doubly linked lists (ver. 1)

• Store the dic-items in a list, in no particular order.• Insert a new dic-item to an arbitrary position of the list,

e.g., the first or last position.• Delete a dic-item using a supplied pointer to it.• Search, and other operations, are implemented by scanning

the list.

Insert, Delete – O(1) timeOther operations – O(n) time

6

Implementing dictionaries using doubly linked lists (ver. 2)

• Store the dic-items in a list, in increasing order of keys.• Insert a new dic-item to the appropriate position• Delete a dic-item by using a supplied pointer to it.• Search is implemented by scanning the list, stopping when

the key of the current item is larger than the key sought.

Insert,Search – O(n) time (or O(n/2) “on average”)

Delete – O(1) timeMin, Max, Successor, Predecessor – O(1) time

7

Implementing dictionaries using (circular) arrays

• Store the dic-items in a list, in increasing order of keys.• Insert a new dic-item to the appropriate position• Delete a dic-item by using a supplied pointer to it.• Search implemented using binary search.

Insert, Delete – O(n) time (or O(n/2) )

Min, Max, Successor, Predecessor – O(1) time

Search – O(log n)

8

Binary search

10 25 38 47 56 67 73 84 95

Successful search:Search(38)

0 1 2 3 4 5 6 7 8

highmid

low

9

Binary searchUnsuccessful search:

Search(39)

10 25 38 47 56 67 73 84 95

0 1 2 3 4 5 6 7 8

highmid

low

10

Binary search

Key k was foundin position mid

Key k should be insertedin position mid or mid+1

Key(Retrieve(L,mid))

11

Can we implement alloperations in O(log n) time?

Yes! Using Binary Search Trees

12

Binary search treesA binary tree in which each node contains a dic-item.

Satisfies the binary-search-tree property:If y is in the left subtree of x, then y.key < x.key.

If y is in the right subtree of x, then y.key > x.key.

2 8

7

5 101right

key

left

x parent item

13

Binary search trees

2 8

7

5 101right

key

left

x parent item

Dic-Item ≡ Tree-NodeWe assume that each Dic-Item now has room for the three additional pointers left, right, parent,

which are initially set to null

D.root

14

A set can be represented using several different trees

1

8

2

7 10

5We assume that all keys are distinctIn most figures, we show the keys and ignore the items.

2 8

7

5 101

Height – length of the longest path from a root to a leaf

height=2

height=4

15

Tree-Search(x,k) – Look for k in the subtree of x

2 8

7

5 101

x

Tree-Search(x,5)

x

x

Search(D,k) Tree-Search(D.root,k)

We usually start the search at the root of the tree:

16

Tree-Position(x,k) – Look for k in the subtree of xReturn the last node encountered

2 8

7

5 101

x

Tree-Position(x,6)

x

x

y

y

y

Returns the node containing 5

Tree-Position(x,k) is used to find insertion points

17

Printing the elements of a BST in sorted order(In-order walk)

2 8

7

5 1010

1

2

3

4

5

Printing, of course, is just an example…

18

Finding the minimum“keep going left”

2 8

7

5 101

19

Successor(x)

If x has a right child, the successor of x is the

minimal element in x.right.x

What if x.right=null ?

“Go right once, and then left all the way”

20

x

Successor(x)

If x.right=null, the successor of x is the lowest ancestor y of x such

that x is in its right subtree

y

“Go up from x untilthe first turn right’’

21

x

Successor(x)

y

If x has the largest key, then Successor(x)=null.

Predecessor is symmetric

22

Insertions and deletions

23

2 8

7

5 101

Insertions

Insert(6)Insert(9)

6 9

24

Binary Search Tree Animations

For the time being, turn all buttons on the right off

http://webdiis.unizar.es/asignaturas/EDA/AVLTree/avltree.html


Warning: There are some differences with what we learn


25

2 8

7

5 101

Deletion: easy cases first

Delete(6) – 6 is a leaf; simply remove it.Delete(8) – 8 has only one child; bypass it.

6 9

Delete(2) – more complicated… Delete(10) – 10 has only one child; bypass it.

26

Deletionmore challenging case

z

y

If z has two children,let y be the successor of z

y has no left childRemove y from the tree

Replace z by y

Binary-search-tree property preserved!

Is it enough to let z.keyy.key?

27

Analysis

Each operation takes O(h+1) time, where h is the height of the tree

In general h may be as large as nWant to keep the tree with small h

28

Balanced trees

h = log2(n+1)−1How do we keep the tree more or less balanced?

A full tree of height h contains n=2h+1 − 1 nodes

29

Randomly built BSTsMaybe balancing will take care of itself?

Not if we insert the elements in sorted order.We get a path of length n

Things are usually ok if weinsert the elements in random order

Theorem: If n distinct keys are inserted into a BST in random order, then the expected depth of each element is O(log n).

Theorem: If n distinct keys are inserted into a BST in random order, then the expected height of the tree is O(log n).

We want worst-case results…

30

Variation: Items only at the leaves

• Keep elements only at the leaves• Each internal node contains a

number to direct the search

5 9

8

7 102

1 2 5 7

8

9 11

Costs space

Internal part similar to The previous tree

31

Analysis

• Each operation takes O(h) time, where h is the height of the tree

• In general h may be as large as n

• Want to keep the tree with small h

32

Balance

h = O(log n)How do we keep the tree balanced through insertions and deletions ?

33

1 )O)n) : Worst Case

אקראיים? איברים n להכניס בממוצע כמה זמן לוקח ( 2

:עץ מושלם (הכי טוב(

:עץ קווי (הכי גרוע(

log n כל איבר לוקח לכל היותר

לצומת(O)n ממוצע של

Data Structures, CS, TAU - 5.6

אנליזת זמן של עץ חיפוש בינארי

n

i

nni1 2

)1( כל האיברים יחד לוקחים

34

:זמן הכנסה ממוצע של עלה בין כל העצים

INSERTרק - ! הסידורים nכל הסידורים של אלמנטים בעץ מתוך -

האפשריים הם שווי הסתברות



nlog

מספר הנחות: הוכחה

עלות איבר = אורך המסלול אליו- עלות ממוצעת = אורך המסלול הממוצע

35

איך נחשב אורך מסלול ממוצע?

n אורך ממוצע של מסלול של צומת שרירותי בעץ בגודל (L)n יהי(אורך מסלול כאן = מס’ צמתים במסלול(

מסוים, מה יהיה אורך המסלול הממוצע?iבהנתן



רקורסיבית:נשתמש בנוסחה

a

i )n-i-1) איברים, יש איבר nלכל עץ בעל ושני תתי עציםaראשון שנסמן

(n-i-1( איברים, בתת העץ השני ישנם iאם בתת העץ הראשון ישנם איברים

1, האורך הוא a האיבר שאנו מחפשים הוא אם(L)i + 1 האורך הואa האיבר שאנו מחפשים קטן מ אם(L)n-i-1 + 1 האורך הואa האיבר שאנו מחפשים גדול מ אם

36

))1(1)(1

())(1(1

)|(

inLnin

iLni

ninL

שמאל מרכזימין

)1(1

)(1

inLnin

iLni



לחיפוש כל איבר היא שווה, ולכן:ההסתברות

היא aשהאיבר שאנו מחפשים הוא ההסתברות

היאaשהאיבר שאנו מחפשים קטן מ ההסתברות

היאaשהאיבר שאנו מחפשים גדול מ ההסתברות

n1

ni

nin 1

אורך המסלול הממוצע הוא:iלכן בהנתן

1

0)|(

1)|(

1

0

1)|(

1

0)(

n

iinL

ninL

n

i ninL

n

iiPnL

a איברים קטנים מ i כהסתברות ש Piנגדיר את



:i עבור כל ערך אפשרי של נסכום

,i + 1 הוא האיבר ה aהסתברות זו שווה להסתברות ש יכול להיות כל אחד מהאיברים בהסתברות שווה, לכןaו

37

niP1

1

0)1()1(

2

11

0)(

2

11

1

0)1(1)(1

1

n

iinLin

n

n

iiLi

n

n

iinL

n

iniLn

in

ולכן:



נשים לב ש:

38

1

0)1()1(

1

0)(

n

iinLin

n

iiLi

1

0)(

2

21)(

n

iiLi

nnL

nnL כי:אינדוקציהכעת אפשר להוכיח ב log41)(

:i < nנניח והתנאי מתקיים עבור כל

1

0log4

2

21

02

21

1

0log41

2

21)(

n

iii

n

n

ii

n

n

iii

nnL



39

1

0log4

2

21

02

21

1

0log41

2

21)(

n

iii

n

n

ii

n

n

iii

nnL

2

21

0

nin

i

1

02

2

2log42

221)(

n

i

iin

nn

nL

11

0

1

02

2

2

log28log2

82log82n

ii

n

i n

n

iin

iin

iin

11

02

2

2

log28log2

82n

ii

n

n

n

nin

in



40

83 ,

8

2121

02

2 ninin

ii n

n

11

022

2

log28log2

82)(n

ii

n

n

n

inn

in

nL

8

3log88

1log822

2

2

2nn

nnn

n

nnL log41)(

, 1loglog2

nn

41

Balance

h = O(log n)How do we keep the tree balanced through insertions and deletions ?- Next chapter!

42

Rotations

x

y

B

C

A

y

x

B

A

C

Right rotate

Left rotate

Documents

Data Structures Hanoch Levi and Uri Zwick March 2011 Lecture 3 Dynamic Sets / Dictionaries Binary Search Trees