1
CSIE, NTUT, TAIWAN
1
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Data Structures
Chuan-Ming Liu
Computer Science & Information Engineering
National Taipei University of Technology
Taiwan
CSIE, NTUT, TAIWAN
2
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Instructor
Chuan-Ming Liu (劉傳銘)
Office: 1530 Technology Building
Computer Science and Information Engineering
National Taipei University of Technology
TAIWAN
Phone: (02) 2771-2171 ext. 4251
Email: [email protected]
Office Hours: Mon: 11:10-12:00, 13:10-14:00 and Thu:10:10 - 12:00, OR by appointment.
CSIE, NTUT, TAIWAN
3
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Teaching Assisant
Bill In-Chi Su (蘇英啟)
Office: 1226 Technology Building
Office Hours: Tue:10:00~12:00 and Wed: 10:00 ~ 12:00, OR by appointment
Email: [email protected]: 02-2771-2171 ext. 4262
2
CSIE, NTUT, TAIWAN
4
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Text Books
Ellis Horowitz, Sartaj Sahni, and Susan Anderson-Frees, Fundamentals of Data
Structures in C, 2nd edition, Silicon Press, 2008.
Supplementary Texts
• Michael T. Goodrich and Roberto Tamassia, Data Structures and Algorithms in JAVA, 4th edition, John Wiley & Sons, 2006. ISBN: 0-471-73884-0.
• Sartaj Sahni, Data Structures, Algorithms, and Applications in JAVA, 2nd edition, Silicon Press, 2005. ISBN: 0-929306-33-3.
• Frank M. Carranno and Walter Savitch, Data Structures and Abstractions with Java, Prentice Hall, 2003. ISBN: 0-13-017489-0.
CSIE, NTUT, TAIWAN
5
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Course Outline
• Introduction and Recursion
• Analysis Tools
• Arrays
• Stacks and Queues
• Linked Lists
• Sorting
• Hashing
• Trees
• Priority Queues
• Search Trees
• Graphs
CSIE, NTUT, TAIWAN
6
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Course Work
• Assignments (50%): 6-8 homework sets and
two LAB projects
• Midterm (20%)
• Final exam (30%)
3
CSIE, NTUT, TAIWAN
7
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Course Policy (1)
• No late homework is acceptable.
• For a regrade please contact me for the question within 10 days from the date when the quiz or exam was officially returned. No regrading after this period.
• Cheating directly affects the reputation of the Department and the University and lowers the morale of other students. Cheating in homework and exam will not be tolerated. An automatic grade of 0 will be assigned to any student caught cheating. Presenting another person's work as your own constitutes cheating. Everything you turn in must be your own doing.
CSIE, NTUT, TAIWAN
8
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Course Policy (2)
• The following activities are specifically forbidden on
all graded course work:
– Theft or possession of another student's solution or partial
solution in any form (electronic, handwritten, or printed).
– Giving a solution or partial solution to another student,
even with the explicit understanding that it will not be
copied.
– Working together to develop a single solution and then
turning in copies of that solution (or modifications) under
multiple names.
CSIE, NTUT, TAIWAN
9
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
First Thing to Do
• Please visit the course web site
http://www.cc.ntut.edu.tw/~cmliu/DS/NTUT_
DS_S09u-GIT/
• Send an email to me using the email
address:[email protected]. I will make a
mailing list for this course. All the
announcements will be broadcast via this
mailing list.
4
CSIE, NTUT, TAIWAN
10
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Introduction
Chuan-Ming Liu
Computer Science & Information Engineering
National Taipei University of Technology
Taiwan
CSIE, NTUT, TAIWAN
11
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Outline
• Data Structures and Algorithms
• Pseudo-code
• Recursion
CSIE, NTUT, TAIWAN
12
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
What is Data Structures
• A data structure * in computer science is a
way of storing data in a computer so that it can
be used efficiently.
– An organization of mathematical and logical
concepts of data
– Implementation using a programming language
– A proper data structure can make the algorithm or
solution more efficient in terms of time and space
*Wikipedia: http://en.wikipedia.org/wiki/Data_structure
5
CSIE, NTUT, TAIWAN
13
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Why We Learn Data Structures
• Knowing data structures well can make our
programs or algorithms more efficient
• In this course, we will learn
– Some basic data structures
– How to tell if the data structures are good or bad
– The ability to create some new and advanced data
structures
CSIE, NTUT, TAIWAN
14
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
What is an Algorithm (1)
An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. All the algorithms must satisfy the following criteria:
– Input
– Output
– Precision (Definiteness)
– Effectiveness
– Finiteness
CSIE, NTUT, TAIWAN
15
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
What is an Algorithm (2)
• Definiteness: each instruction is clear and
unambiguous
• Effectiveness: each instruction is
executable; in other words, feasibility
• Finiteness: the algorithm terminates after
a finite number of steps.
6
CSIE, NTUT, TAIWAN
16
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
What is an Algorithm (3)
Input OutputDefiniteness
Effectiveness
Finiteness
Computational Procedures
CSIE, NTUT, TAIWAN
17
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Procedures vs. Algorithms
• Termination or not
• One example for procedure is OS
• Program, a way to express an algorithm
CSIE, NTUT, TAIWAN
18
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Expressing Algorithms
• Ways to express an algorithm
– Graphic (flow chart)
– Programming languages (C/C++)
– Pseudo-code representation
7
CSIE, NTUT, TAIWAN
19
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Outline
• Data Structures and Algorithms
• Pseudo-code
• Recursion
CSIE, NTUT, TAIWAN
20
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Example – Selection Sort
Suppose we must devise a program that sorts a
collection of n≥1 elements.
Idea: Among the unsorted elements, select the
smallest one and place it next in the sorted list.
for (int i=1; i<=n; i++) {
examine a[i] to a[n] and suppose
the smallest element is at a[j];
interchange a[i] and a[j];
}
CSIE, NTUT, TAIWAN
21
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Pseudo-code for Selection Sort
SelectionSort(A)
/* Sort the array A[1:n] into nondecreasing order. */
for i ←1 to length[A]
do j ← i
for k← (i+1) to length[A]
do if A[k]<A[j]
then j ← k;
t← A[i]
A[i] ← A[j]
A[j] ← t
8
CSIE, NTUT, TAIWAN
22
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Maximum of Three Numbers
This algorithm finds the largest of the numbers
a, b, and c.
Input Parameters: a, b, c
Output Parameter: x
max(a,b,c,x) {
x = a
if (b > x) // if b is larger than x, update x
x = b
if (c > x) // if c is larger than x, update x
x = c
}
CSIE, NTUT, TAIWAN
23
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Pseudo-Code Conventions
• Indentation as block structure
• Loop and conditional constructs similar to
those in PASCAL, such as while, for,
repeat( do – while) , if-then-else
• // as the comment in a line
• Using = for the assignment operator
• Variables local to the given procedure
CSIE, NTUT, TAIWAN
24
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Pseudo-Code Conventions
• Relational operators: ==, != , ≥, ≤.
• Logical operators: &&, ||, !.
• Array element accessed by A[i] and A[1..j] as
the subarray of A
9
CSIE, NTUT, TAIWAN
25
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Outline
• Data Structures and Algorithms
• Pseudo-code
• Recursion
CSIE, NTUT, TAIWAN
26
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Recursion
• Recursion is the concept of defining a method
that makes a call to itself
• A method calling itself is making a recursive
call
• A method M is recursive if it calls itself (direct
recursion) or another method that ultimately
leads to a call back to M (indirect recursion)
CSIE, NTUT, TAIWAN
27
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Repetition
• Repetition can be achieved by
– Loops (iterative) : for loops and while loops
– Recursion (recursive) : a function calls itself
• Factorial function
– General definition: n! = 1� 2� 3� ��� � (n-1)� n
– Recursive definition
−⋅
==
elsenfn
nnf
)1(
0 if1)(
10
CSIE, NTUT, TAIWAN
28
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Method for Factorial Function
• recursive factorial function
public static public static public static public static intintintint recursiveFactorial(intintintint n) {if if if if (n == 0) return return return return 1;else return else return else return else return n * recursiveFactorial(n- 1);
}
CSIE, NTUT, TAIWAN
29
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Content of a Recursive Method
• Base case(s)
– Values of the input variables for which we perform no
recursive calls are called base cases (there should be at
least one base case).
– Every possible chain of recursive calls must eventually
reach a base case.
• Recursive calls
– Calls to the current method.
– Each recursive call should be defined so that it makes
progress towards a base case.
CSIE, NTUT, TAIWAN
30
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Visualizing Recursion
• Recursion trace
• A box for each
recursive call
• An arrow from each
caller to callee
• An arrow from each
callee to caller
showing return value
Example recursion trace:
recursiveFactorial(4)
recursiveFactorial(3)
recursiveFactorial(2)
recursiveFactorial(1)
recursiveFactorial(0)
return 1
return 1*1
return 2*1
return 3*2
return 4*6
11
CSIE, NTUT, TAIWAN
31
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
About Recursion
• Advantages
– Avoiding complex case analysis
– Avoiding nested loops
– Leading to a readable algorithm description
– Efficiency
• Examples
– File-system directories
– Syntax in modern programming languages
CSIE, NTUT, TAIWAN
32
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Linear Recursion
• The simplest form of recursion
• A method M is defined as linear recursion if it
makes at most one recursive call
• Example: Summing the Elements of an Array
– Given: An integer array A of size m and an integer
n, where m≧n≧1.
– Problem: the sum of the first n integers in A.
CSIE, NTUT, TAIWAN
33
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Summing the Elements of an Array
• Solutions:
– using (for) loop
– using recursion
Algorithm LinearSum(A, n):
Input: integer array A, an integer n ≧1, such that A has at least n elements
Output: The sum of the first n integers in A
if n = 1 then
return A[0]
else
return LinearSum(A, n - 1) + A[n - 1]
Note: the index starts from 0.
12
CSIE, NTUT, TAIWAN
34
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Recursive Method
• An important property of a recursive method –
the method terminates
• An algorithm using linear recursion has the
following form:
– Test for base cases
– Recur
CSIE, NTUT, TAIWAN
35
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Analyzing Recursive Algorithms
• Recursion trace
– Box for each instance of the method
– Label the box with parameters
– Arrows for calls and returnsLinearSum (A,5)
LinearSum (A,1)
LinearSum(A,2)
LinearSum (A,3)
LinearSum(A,4)
call
call
call
call return A[0] = 4
return 4 + A[1] = 4 + 3 = 7
return 7 + A[2] = 7 + 6 = 13
return 13 + A[3] = 13 + 2 = 15
call return 15 + A[4] = 15 + 5 = 20
CSIE, NTUT, TAIWAN
36
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Example
• Reversing an Array by Recursion
– Given: An array A of size n
– Problem: Reverse the elements of A (the first
element becomes the last one, …)
• Solutions
– Nested loop ?
– Recursion
13
CSIE, NTUT, TAIWAN
37
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Reversing an Array
Algorithm ReverseArray(A, i, j):
Input: An array A and nonnegative integer indices iand j
Output: The reversal of the elements in A starting at index i and ending at j
if i < j then
Swap A[i] and A[ j]
ReverseArray(A, i + 1, j - 1)
return
CSIE, NTUT, TAIWAN
38
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Facilitating Recursion
• In creating recursive methods, it is important
to define the methods in ways that facilitate
recursion.
• This sometimes requires we define additional
parameters that are passed to the method.
– For example, we defined the array reversal method
as ReverseArray(A, i, j), not ReverseArray(A).
CSIE, NTUT, TAIWAN
39
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Example – Computing Powers
• The power function, p(x, n)=xn, can be
defined recursively:
• Following the definition leads to an O(n) time
recursive algorithm (for we make n recursive
calls).
• We can do better than this, however.
−⋅
==
else)1,(
0 if1),(
nxpx
nnxp
14
CSIE, NTUT, TAIWAN
40
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Recursive Squaring
• We can derive a more efficient linearly recursive algorithm by using repeated squaring:
• For example,
24 = 2(4/2)2 = (24/2)2 = (22)2 = 42 = 16
25 = 21+(4/2)2 = 2(24/2)2 = 2(22)2 = 2(42) = 32
26 = 2(6/2)2 = (26/2)2 = (23)2 = 82 = 64
27 = 21+(6/2)2 = 2(26/2)2 = 2(23)2 = 2(82) = 128
>
>
=
−⋅=
even is 0 if
odd is 0 if
0 if
)2/,(
)2/)1(,(
1
),(2
2
n
n
n
nxp
nxpxnxp
CSIE, NTUT, TAIWAN
41
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
A Recursive Squaring Method
Algorithm Power(x, n):
Input: A number x and integer n ≧ 0
Output: The value xn
if n = 0 then
return 1
if n is odd then
y = Power(x, (n - 1)/ 2)
return x · y ·y
else /* n is even */
y = Power(x, n/ 2)
return y · y
CSIE, NTUT, TAIWAN
42
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing LabAnalyzing the Recursive
Squaring Method
Algorithm Power(x, n):
Input: A number x and integer n ≧0
Output: The value xn
if n = 0 then
return 1
if n is odd then
y = Power(x, (n - 1)/ 2)
return x � y � y
else
y = Power(x, n/ 2)
return y � y
Each time we make a recursive call we halve the value of n; hence, we make log n recursive calls. That is, this method runs in O(log n) time.
It is important that we used a variable twice here rather than calling the method twice.
15
CSIE, NTUT, TAIWAN
43
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Tail Recursion
• Tail recursion occurs when a linearly
recursive method makes its recursive call as
its last step.
• Such methods can be easily converted to
non-recursive methods (which saves on
some resources).
• The array reversal method is an example.
CSIE, NTUT, TAIWAN
44
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Example – Using Iteration
Algorithm IterativeReverseArray(A, i, j ):
Input: An array A and nonnegative integer
indices i and j
Output: The reversal of the elements in A
starting at index i and ending at j
while i < j do
Swap A[i ] and A[ j ]
i = i + 1
j = j - 1
return
CSIE, NTUT, TAIWAN
45
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Binary Recursion
• Binary recursion occurs whenever there are
two recursive calls for each non-base case.
• Example: BinaySum
16
CSIE, NTUT, TAIWAN
46
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing LabExample – Summing n
Elements in an Array
• Recall that this problem has been solved using
linear recursion
• Using binary recursion instead of linear recursion
Algorithm BinarySum(A, i, n):
Input: An array A and integers i and n
Output: The sum of the n integers in A starting at index i
if n = 1 then
return A[i ]
return BinarySum(A,i,n/2)+BinarySum(A,i+n/2,n/2)
CSIE, NTUT, TAIWAN
47
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Recursion Trace
• Note the floor and ceiling used in the method
3, 1
2, 2
0, 4
2, 11, 10, 1
0, 8
0, 2
7, 1
6, 2
4, 4
6, 15, 1
4, 2
4, 1
CSIE, NTUT, TAIWAN
48
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Fibonacci Numbers
• Fibonacci numbers are defined recursively:
F0 = 0
F1 = 1
Fi = Fi-1+ Fi-2 for i > 1.
• Example: 0, 1, 1, 2, 3, 5, 8, …
17
CSIE, NTUT, TAIWAN
49
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing LabFibonacci Numbers
– Binary Recursion
Algorithm BinaryFib(k):
Input: Nonnegative integer k
Output: The kth Fibonacci number Fk
if k≦ 1 then
return k
else
return BinaryFib(k-1) + BinaryFib(k-2)
CSIE, NTUT, TAIWAN
50
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Analyzing the Binary Recursion
• Algorithm BinaryFib makes a number of calls
that are exponential in k
• By observation, there are many redundant
computations:
F0 = 0; F1 = 1; F2 = F1+ F0;
F3 = F2+ F1 =(F1
+ F0)+F1; …
• The above two results show the inefficiency of
the method using binary recursion
CSIE, NTUT, TAIWAN
51
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
A Better Fibonacci Algorithm
• Using linear recursion instead – avoid the redundant computation:Algorithm LinearFibonacci(k):
Input: A nonnegative integer k
Output: Pair of Fibonacci numbers (Fk, Fk-1)
if k = 1 then
return (k, 0)
else
(i, j) = LinearFibonacci(k - 1)
return (i +j, i)
• Runs in O(k) time.
18
CSIE, NTUT, TAIWAN
52
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Multiple Recursion
• Motivating example: summation puzzles
• pot + pan = bib
• dog + cat = pig
• boy + girl = baby
• Multiple recursion: makes potentially many
recursive calls (not just one or two).
CSIE, NTUT, TAIWAN
53
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Algorithm for Multiple Recursion
Algorithm PuzzleSolve(k,S,U):
Input: An integer k, sequence S, and set U (the universe of elements to test)
Output: An enumeration of all k-length extensions to S using elements in U
without repetitions
for all e in U do
Remove e from U {e is now being used}
Add e to the end of S
if k = 1 then
Test whether S is a configuration that solves the puzzle
if S solves the puzzle then
return “Solution found: ” S
else
PuzzleSolve(k - 1, S,U)
Add e back to U {e is now unused}
Remove e from the end of S
CSIE, NTUT, TAIWAN
54
Applied Computing LabApplied Computing LabApplied Computing LabApplied Computing Lab
Visualizing PuzzleSolve()
PuzzleSolve (3,() ,{a,b,c})
Initial call
PuzzleSolve (2,c,{a,b})PuzzleSolve (2,b,{a,c})PuzzleSolve (2,a,{b,c})
PuzzleSolve (1,ab ,{c} )
PuzzleSolve (1,ac ,{b}) PuzzleSolve (1,cb ,{a})
PuzzleSolve (1,ca ,{b})
PuzzleSolve (1,bc ,{a})
PuzzleSolve (1,ba ,{c})
abc
acb
bac
bca
cab
cba