COSC 3101N J. Elder

Announcements• Midterms are marked

• Assignment 2: – Still analyzing

COSC 3101N J. Elder

Loop Invariants (Revisited)• Question 5(a): Design an iterative

algorithm for Parity:

p=0for i = 1 to n do p = xor(p,s[i])return p

Loop Invariant?

LI: After i iterations are performed, p=Parity(s[1…i])

COSC 3101N J. Elder

What is a loop invariant?

• An assertion about the state of one or more variables used in the loop.

• When the exit condition is met, the LI leads naturally to the postcondition (the goal of the algorithm).

• Thus the LI must be a statement about the variables that store the results of the computation.

COSC 3101N J. Elder

Know What an LI Is Not

``the LI is NOT...''– code

– The steps taken by the algorithm

– A statement about the range of values assumed by the loop index.

COSC 3101N J. Elder

Dynamic Programming: Recurrence

COSC 3101N J. Elder

Dynamic programming

• Step 1: Describe an array of values you want to compute.

• Step 2: Give a recurrence for computing later values from earlier (bottom-up).

• Step 3: Give a high-level program.

• Step 4: Show how to use values in the array to compute an optimal solution.

COSC 3101N J. Elder

Example 1. Rock climbing

At every step our climber can reach exactly three handholds: above, above and to the right and above and to the left.

There is a table of “danger ratings” provided. The “Danger” of a path is the sum of danger ratings of all handholds on the path.

5 3

4

2

COSC 3101N J. Elder

For every handhold, there is only one “path” rating. Once we have reached a hold, we don’t need to know how we got there to move to the next level.

This is called an “optimal substructure” property. Once we know optimal solutions to

subproblems, we can compute an optimal solution to the problem itself.

COSC 3101N J. Elder

Step 2. Define a RecurrenceLet C(i,j) represent the danger of hold (i,j)

Let A(i,j) represent the cumulative danger of the safest path from the bottom to hold (i,j)

ThenA(i,j) = C(i,j)+min{A(i-1,j-1),A(i-1,j),A(i-1,j+1)}

i.e., the safest path to hold (i,j) subsumes the safest path to holds at level i-1 from which hold (i,j) can be reached.

COSC 3101N J. Elder

Example 2. Activity Scheduling with Profits

1 1g 2 10g

3 1g

COSC 3101N J. Elder

Step 2. Provide a Recurrent Solution

(0) 0( ) max{ ( 1), ( ( ))}, {1,..., }i

AA i A i g A H i i n

Decide not to schedule activity i

Profit from scheduling activity i

Optimal profit from scheduling activities that end before activity i begins

1 21. Sort activities according to fini ( (shing lo ti g ))me: nf f O nf n

2. {1,..., }, compute ( ) max{ {1,2,..., 1 ( ( log )} } )| l ii n H Oi l i f s n n

. . ( ) is the last event that ends before event starts.i e H i i

COSC 3101N J. Elder

Example 3: Scheduling Jobs with Deadlines, Profits and Durations

information ( , , ) about activities, whereinteger deadline for job integer duration of job real-valued pr

Input

ofit of job

:

i i i

i

i

i

d t g nd it ig i

A C is a sequence { (1), (2),..., ( )} such that: ( ) scheduled start time of job ( ) 1 if job is not scheduled

schedule C C C C nC i iC i i

A feasible schedule C with maximum profit: (Output ): ii C

P C g

A schedule C is if each scheduled job finishes by its deadlineand no two scheduled jobs over

feasilap

bleped.

COSC 3101N J. Elder

Dynamic Programming Solution

Precomputation:

1 2Sort activities according to de ( ( log ))adline: nd Od n nd

COSC 3101N J. Elder

Step 2. Provide a Recurrent Solution(0, ) 0 {0,..., }

For {1,..., }, {0,..., }, define t min{ , }( is the latest time that we can schedule job so that itends both by its deadline and by time .)Then:

0 ( , ) ( 1, )

i i

A t t d

i n t d t d tt i

t

t A i t A i tt

0 ( , ) max{ ( 1, ), ( 1, )}iA i t A i t g A i t

Decide not to schedule job i

Profit from job i Profit from scheduling activities that end before job i begins

COSC 3101N J. Elder

Step 2. (cntd…) Proving the Recurrent Solution

We effectively schedule job i at the latest possible time.

This leaves the largest and earliest contiguous block of time for scheduling jobs with earlier deadlines.

For {1,..., }, {0,..., }, define t min{ , }( is the latest time that we can schedule job so that itends both by its deadline and by time .)

i ii n t d t d tt i

t

Then:0 ( , ) ( 1, )t A i t A i t

0 ( , ) max{ ( 1, ), ( 1, )}it A i t A i t g A i t

COSC 3101N J. Elder

2d 1id id1d … t

it

t

event iCase 1

2d 1id id1d … t

it

t

event iCase 2

COSC 3101N J. Elder

Example 3. Longest Common Subsequence

COSC 3101N J. Elder

Optimal Substructure• Input: 2 sequences, X = x1, . . . , xm and Y =

y1, . . . , yn.

COSC 3101N J. Elder

Proof of Optimal Substructure Part 1

1 1

1 1

1

{ ,..., }; { ,..., }{ ,..., }; { ,..., }

{ ,..., } is an LCS of an

Notati

on:

d .

m n

i i i i

k

X x x Y y yX x x Y y yZ z z X Y

1 1 11. If , then and is an LCS of and . :

m n k m n k m n

Theorex y z x y Z X

mY

Proof that :k m nz x y Suppose . Then could append to common subsequenceof and of length 1 contradiction.

k m m nz x x y ZX Y k

1 1 1Proof that is an LCS of and :k m nZ X Y

1 1 1 is a length-( 1) common subsequence of and .k m nZ k X Y

1 1Suppose there is a longer common subsequence of and .m nW X Y

Appending to common subsequence of length >k contradictionm nx y W

COSC 3101N J. Elder

Proof of Optimal Substructure Part 2

1 1

1 1

1

{ ,..., }; { ,..., }{ ,..., }; { ,..., }

{ ,..., } is an LCS of an

Notati

on:

d .

m n

i i i i

k

X x x Y y yX x x Y y yZ z z X Y

12. If and , then is an LCS of :

an d .m n k m mx y z x ZTheor

Ym

Xe

Proof :

1 is a common subsequence of and .k m mz x Z X Y

1Suppose there is a common subsequence of and mW X Y

of length also a common subsequence of and contradictionk W X Y

COSC 3101N J. Elder

Proof of Optimal Substructure Part 3

1 1

1 1

1

{ ,..., }; { ,..., }{ ,..., }; { ,..., }

{ ,..., } is an LCS of an

Notati

on:

d .

m n

i i i i

k

X x x Y y yX x x Y y yZ z z X Y

12. If and , then is an LCS of a:

nd .m n k n nx y zTheor

y Z X Yem

Proof :

As for Part 2.

COSC 3101N J. Elder

Step 2. Provide a Recurrent Solution

Input sequences are empty

Last elements match: must be part of LCS

Last elements don’t match: at most one of them is part of LCS

COSC 3101N J. Elder

Example 6: Longest Increasing Subsequence

• Input: 1 sequence, X = x1, . . . , xn.

• Output: the longest increasing subsequence of X.

• Note: A subsequence doesn’t have to be consecutive, but it has to be in order.

COSC 3101N J. Elder

Step 1. Define an array of values to compute

{0,..., }, length of LIS of endiA( ng in .) iii n X x

Ultimately, we are intere maxsted { ( }in .) |1A i i n

COSC 3101N J. Elder

Step 2. Provide a Recurrent Solution

for 1 ,( ) 1 max{ ( ) |1 and }j i

i nA i A j j i x x

COSC 3101N J. Elder

Step 3. Provide an Algorithm

Running time? O(n2)function A=LIS(X)

for i=1:length(X)

m=0;

for j=1:i-1

if X(j) < X(i) & A(j) > m

m=A(j);

end

end

A(i)=m+1;

end

COSC 3101N J. Elder

Step 4. Compute Optimal Solution

Running time? O(n)

function lis=printLIS(X, A)[m,mi]=max(A);lis=printLISm(X,A,mi,'LIS: ');lis=[lis,sprintf('%d', X(mi))];

function lis=printLISm(X, A, mi, lis)if A(mi) > 1 i=mi-1; while ~(X(i) < X(mi) & A(i) == A(mi)-1) i=i-1; end lis=printLISm(X, A, i, lis); lis=[lis, sprintf('%d ', X(i))];end

COSC 3101N J. Elder

LIS Example

X = 96 24 61 49 90 77 46 2 83 45

A = 1 1 2 2 3 3 2 1 4 2

> printLIS(X,A)

> LIS: 24 49 77 83

COSC 3101N J. Elder

Example 6: Optimal Binary Search Trees

1 1 2

Sequence { ,..., } of distinct keys,

probability

Inp

that a search is for key

ut:

n n

i i

K k k n k k kp k

BST with minimum expected searcOutp

h ut:

cost

COSC 3101N J. Elder

Expected Search Cost

Which BST is more efficient?

Cost = # of items examined.

For key , cost = depth ( ) 1i T ik k

1

[search cost in ]

(depth ( ) 1)n

T i ii

E T

k p

1

1 depth ( )n

T i ii

k p

COSC 3101N J. Elder

Observations

COSC 3101N J. Elder

Optimal Substructure

COSC 3101N J. Elder

Optimal Substructure (cntd…)

TLet e expected search cost in subtree , starting from root( )T T

depth ( )j

l T ll i

p k

TLet e expected search cost in subtree , starting from root( )T T

(depth (root( )) depth ( ))j

l T T ll i

p T k

depth (root( )) depth ( )j j

T l l T ll i l i

T p p k

Tdepth (root( )) ej

T ll i

T p

T

T

COSC 3101N J. Elder

Recursive Solution

COSC 3101N J. Elder

Recursive Solution (cntd…)

COSC 3101N J. Elder

Step 2. Provide a Recurrent Solution

Expected cost of search for left subtree

Added cost when subtrees embedded under root

Expected cost of search for right subtree

COSC 3101N J. Elder

n)

Step 3. Provide an Algorithm

Running time? O(n3)

work on subtrees of increasing size l

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example (cntd…)

COSC 3101N J. Elder

Step 4. Compute Optimal Solution

Running time? O(n)

COSC 3101N J. Elder

Elements of Dynamic Programming

• Optimal substructure: – an optimal solution to the problem contains

within it optimal solutions to subproblems.

COSC 3101N J. Elder

Elements of Dynamic Programming

• Cut and paste: prove optimal substructure by contradiction:– assume an optimal solution to a problem with suboptimal solution to

subproblem

– cut out the suboptimal solution to the subproblem.

– paste in the optimal solution to the subproblem.

– show that this results in a better solution to the original problem.

– This contradicts our assertion that our original solution is optimal.

COSC 3101N J. Elder

• Dynamic programming uses optimal substructure from the bottom up:– First find optimal solutions to subproblems

– Then choose which to use in optimal solution to problem.

Elements of Dynamic Programming

COSC 3101N J. Elder

Section V. Graph Algorithms

COSC 3101N J. Elder

(c) The subgraph of the graph in part (a) induced by the vertex set {1,2,3,6}.

(a) A directed graph G = (V, E), where V = {1,2,3,4,5,6} and E = {(1,2), (2,2), (2,4), (2,5), (4,1), (4,5), (5,4), (6,3)}. The edge (2,2) is a self-loop.

(b) An undirected graph G = (V,E), where V = {1,2,3,4,5,6} and E = {(1,2), (1,5), (2,5), (3,6)}. The vertex 4 is isolated.

Directed and Undirected Graphs

COSC 3101N J. Elder

Graph Isomorphism

COSC 3101N J. Elder

Trees

COSC 3101N J. Elder

Representations: Undirected Graphs

Adjacency List Adjacency Matrix

COSC 3101N J. Elder

Representations: Directed Graphs

Adjacency List Adjacency Matrix