Download ppt - 5 searching

Searching in a Graph

Jeff Edmonds

York University COSC 3101LectureLecture 55

Generic SearchBreadth First Search

Dijkstra's Shortest Paths AlgorithmDepth First Search

Linear OrderNetwork Flow

Linear Programming

Graph

a

c

b Node ~ city or computer

Edge ~ road or network

Undirected or Directed

A surprisingly large number of problems in computer science can be expressed as a graph theory problem.

Graph Search

Specification: Reachability-from-single-source s•Preconditions: The input is a graph G (either directed or undirected) and a source node s. •Postconditions: Output all the nodes u that are reachable by a path in G from s.

Graph Search

Basic Steps:

su

Suppose you know that u is reachable from s

v

& there is an edge from u to v

You know that v is reachable from s

Build up a set of reachable nodes.

Graph Search

s

reachable

fuk reachable

dvw reachable

ehd reachable

bt

reachable

dvw reachable

ehd reachable

bt

•How do we keep track of all of this information?

•How do we avoid cycling?

sa

c

hk

fi

lm

j

eb

gd

Graph Search

sa

c

hk

fi

lm

j

eb

gd

Graph Search

sa

c

hk

fi

lm

j

eb

gd

Graph Search

sa

c

hk

fi

m

j

eb

gd

We know found nodes are reachable from s because we have traced out a path.

If a node has been handled, then all of its neighbors have been found.

Graph Search

l

sa

c

hk

fi

m

j

eb

gd



Graph Search

l

Handle some foundNotHandled nodei.e. find its neighborsDon’t refind a node.



Graph Search

Handle some foundNotHandled nodei.e. find its neighbors

sa

c

hk

fi

m

j

eb

gd

l



Graph Search

# of found nodes.

sa

c

hk

fi

m

j

eb

gd

l

measureprogress

# of handled nodes.Might not increase.



Graph Search

Node s is foundNotHandled

sa

c

hk

fi

lm

j

eb

gd

Other nodes NotFound



Graph Search

All nodes found.Exit

a

c

hk

fi

m

j

eb

gd

l

When can’t make any more progress.No. Might not find all.

When all found nodes are have been handled.Handle some foundNotHandled node

s

Graph Search



a

c

hk

fi

m

j

eb

gd

lExit All found nodes are handled.

Exit

•Postconditions: Output all the nodes u that are reachable by a path in G from s.

Output Found nodes



Graph Search

Exit

a

c

hk

fi

m

j

eb

gd

lAll found nodes are handled.

Exit Found nodes are reachable from s. Reachable nodes have been Found.

•Postconditions:



Graph Search

Exit

a

c

hk

fi

m

j

eb

gd

lAll found nodes are handled.

Exit Reachable nodes have been Found. •Postconditions:

found = handled

handled

Notfound

[A B] = [B A]NotFound nodes not reachable.

Graph SearchSpecification of Reachability-from-single-source s

•Preconditions: The input is a graph G (either directed or undirected) and a source node s. •Postconditions: Output all the nodes u that are reachable by a path in G from s.

EndingInitial ConditionsMake Progress

Maintain Loop InvDefine Exit ConditionDefine Step

Define Measure of Progress

Define Loop Invariants

Define Problem

km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

Graph Search# of handled nodes.

Handle some foundNotHandled node

Time = O(n)

fuk

O(n) iterations, but iteration takes more than O(1)

O(n) neighbors= O(n2)Could be fewer?

Each edge visited, times.2= O(E)

Size = O(E)Linear time.

Graph Search


•Queue: Handle in order found.

•Breadth-First Search

•Stack: Handle most recently found

•Depth-First Search

•Priority Queue: Handle node that seems to be closest to s.

•Shortest (Weighted) Paths:

Breadth First Search

Pre & Post Conditions

Algorithm

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

s

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

a

bgd

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bgd

cf

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bg

cfme

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bcfmej

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

cfmej

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

fmejh

i

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

mejh

i

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

ejh

il

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

jh

il

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

h

il

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

ilk

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

lk

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

k

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

BFS

What order are the nodes found?

i.e. Breadth First Search

Algorithm

So far, the nodes have been found in order of length from s.

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

s

d=0

d=0

BFS FoundNot Handled

Queue

d=0a

bgd

d=1

s

a

c

h

k

f

i

l

m

j

e

b

gd

d=0d=1

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

a

bgd

d=0d=1

d=1

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bgd

cf

d=0d=1

d=2

d=1

d=2

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bg

cfme

d=0d=1

d=2

d=1

d=2

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queued=0

d=1

d=2

b

j

cfme

d=1

d=2

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queued=0

d=1

d=2

j

cfme

d=1

d=2

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

cfmej

d=0d=1

d=2

d=2

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

fmejh

i

d=0d=1

d=2

d=3

d=2

d=3

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

mejh

i

d=0d=1

d=2

d=3

d=2

d=3

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

ejh

il

d=0d=1

d=2

d=3

d=2

d=3

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

jh

il

d=0d=1

d=2

d=3

d=2

d=3

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

h

il

d=0d=1

d=2

d=3

d=2

d=3

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

h

d=0d=1

d=2

d=3

il

d=3

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

ilk

d=0d=1

d=2

d=3d=4

d=3

d=4

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

lk

d=0d=1

d=2

d=3d=4

d=3

d=4

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

k

d=0d=1

d=2

d=3d=4

d=3

d=4

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

k

d=0d=1

d=2

d=3d=4

d=4

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queued=0

d=1

d=2

d=3d=4

d=4d=5

BFS



Post Conditions: Finds a shortest path from sto each node v and its length.

To prove path is shortest:Prove there is a path of this length.Prove there are no shorter paths.

Give a path(witness)

Basic Steps:

su

The shortest path to uhas length d

v


There is a path to v with length d+1.

BFS

BFS



Post Conditions: Finds a shortest path from sto each node v and its length.

When we find v, we know there isn't a shorter path to it because ?

Prove there are no shorter paths.

Otherwise, we would have found it already.

BFSData structure for storing tree: For each node v, store (v) to be parent of v.

Basic Steps:

su

Path to u

v


Parent of v is

(v)

(v) = u.

BFS

Graph Search








Dijkstra's Shortest-Weighted Paths

Specification: Dijkstra's Shortest-Weighted Paths Reachability-from-single-source s

•Preconditions: The input is a graph G (either directed or undirected) with positive edge weights and a source node s. •Postconditions: Finds a shortest weighted path from s to each node v and its length.

Dijkstra's Shortest-Weighted Paths

s

u

v

100

11

1

1w

r

Length of shortest path from s to u?

4


BFS

s

u

v

100

11

1

1w

r

Which node is found first?

Is the same true for Dijkstra's Alg?


s

u

v

100

11

1

1w

r

It has the longest path from s.Which node is found first?

Is the same true for Dijkstra's Alg?

BFS


s

u

v

100

11

1

1w

r

In what order do we handle the FoundNotHandled nodes?

handled

Dijkstra's

Handle node that seems to be closest to s.


Give a path(witness)

Dijkstra'sSo far, the nodes have been handled in order of length from s.

Post Conditions: Finds a shortest weighted path from s to each node v and its length.


Dijkstra's

When we handle v, we know there isn't a shorter path to it because ?

Otherwise, we would have handled it already.

Post Conditions: Finds a shortest weighted path from s to each node v and its length.

So far, the nodes have been handled in order of length from s.

Basic Steps:

u

The shortest of handled paths to u has length d(u)

Dijkstra's


w<u,v>

vs

The shortest of handled paths to v has length d(v)

The shortest known path to v has length

min( d(v), d(u)+w<u,v> ).

Handle node that seems to be closest to s.Need to keep approximate shortest distances.

…

Dijkstra'sThe shortest of handled paths

to v has length d(v)

uvs

Dijkstra's

uvs

Handled nodes Found nodes

Handled Edges?


Dijkstra's

uvs

Handled Nodes Found Nodes

Handled Edges


Handled Paths?

Dijkstra's

uvs

Handled Nodes Found Nodes

Handled Edges


Handled Paths

Dijkstra's

uvs

Handled Nodes


d(v) is the length of the shortest handled paths to v.

For handled u,d(u) is the length of the

shortest paths to u.

NotFound

d(v’)=

Exit

Dijkstra'sFor handled w,

d(w) is the length of the shortest paths to w.

Handle node with smallest d(u).

For handled u, d(u) is the length of the shortest paths to u.

d(v) is the length of the shortest handled

path to v.





d(u) is the length of the shortest handled

path to u.

d(u)





d(u)

d(u) is the length of the shortest handled

path to u.

First not handled node in path.





d(u)

First not handled node in path.d(u’)

d(u) & d(u’) are the length of the shortest handled

paths to u and u’.

Dijkstra'sd(v) is the length

of the shortest handled path to v.

For handled w,d(w) is the length of the

shortest paths to w.


d(v) is the length of the shortest handled path to v.

d(u)

d(v)

w<u,v>

d(v) = min( d(v), d(u)+w<u,v> ).

Time = O(E)Each edge visited, times.2

?

Dijkstra's

This number of times following an edge.

Must update Priority Queue.

Takes time O(log(n))

Time = O(E log(n))

For each update, d(v) = min( d(v), d(u)+w<u,v> ).

Heap

Dijkstra's Shortest-Weighted PathsSpecification: Dijkstra's Shortest-Weighted Paths Reachability-from-single-source s

•Preconditions: The input is a graph G (either directed or undirected) with positive edge weights and a source node s. •Postconditions: Finds a shortest weighted path from s to each node v and its length.

EndingInitial ConditionsMake Progress

Maintain Loop InvDefine Exit ConditionDefine Step

Define Measure of Progress

Define Loop Invariants

Define Problem

km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

Dijkstra's

Dijkstra's

Graph Search








DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Stack

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Stack

s

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Stack

a

bgd

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Stack

a

mgd

e

DFS

DFS

The nodes in the stack form a path starting at s.

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Stack <node,# edges>

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled


s,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1h,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1h,1k,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1h,1

Tree Edge

Path on Stack

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,1

Cross Edgeto handled node

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,2

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,3l,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,3l,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,3

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,1

Back Edgeto node on Stack

Forms a cycle

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,2m,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,2m,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,2

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,5f,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,5f,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,5

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,3

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,2

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,2

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,3e,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,3e,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,3

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,3

FoundNot Handled


DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,0

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,1

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,2

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,3

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled


Linear Order

Ingredients:•Instances: The possible inputs to the problem. •Solutions for Instance: Each instance has an exponentially large set of solutions. •Cost of Solution: Each solution has an easy to compute cost or value.

Specification•Preconditions: The input is one instance.•Postconditions: An valid solution with optimal cost. (minimum or maximum)

Optimization Problems

Specification: Network Flow•Preconditions: The input is a network G directed edges with capacities,a source node s, and a sink node t. •Postconditions: Finds a Maximum Flow F.

Network Flow

Network Flow

No leaks, not extra flow

Value of flow = flow out of s = flow into t = flow across any cut

Network Flow

Network Flow

•Given Flow F•Construct Augmenting Graph GF

•Find path P using BFS, DFS, or generic search algorithm•Let w be the max amount flowcan increase along path P. •Increase flow along path P by w. i.e newF = oldF + w × P

Network Flow

Stop

Hill ClimbingWe have a valid solution.(not necessarily optimal)

Take a step that goes up.

measure

progressValue of our solution.

Problems:

Exit Cant take a step that goes up.

Running time?

Initially have the “zero” solution

Local Max

Global Max

Can our Network Flow Algorithm get stuck in a local maximum?

Make small local changes to your solution toconstruct a slightly better solution.

Network Flow

Previous Input

Previous Output

Same Input

Yes! Our Network Flow Algorithm can get stuck.

Worse Output

Need only one example.

Network Flow

Network Flow

Network Flow

Network Flow

Network Flow

Previous Input

Previous Output

Same Input


Worse Output

Need only one example.

Hill ClimbingAvoiding getting stuck

in a local maximum

Good ExecutionBad Execution

•Made better choices of direction•Hard

•Back up an retry•Exponential time

•Define a bigger step

Hill ClimbingDifferent Solutions

Current Solution

Alg defines to where alg can stepi.e. what small local

changes can be made to current solution

This defines the topography

Hill ClimbingDifferent Solutions

Current Solution

Define a slightlybigger step

This defines the topography

Perhaps removes some local maxima

Network Flow

Previous Input

Previous Output

Same Input


Worse Output

Need only one example.Mistake?

Putting 2 through this edge

Network Flow

u v0/75

0/10

Flow Graph

u v75

10

Augmentation Graph75-21=54

21+10=31

u v21/75

10

Flow Graph

u v

Augmentation Graph

2

Network Flow

Network Flow

Network Flow

Network Flow

Network Flow

Network Flow

Hill Climbing

Network Flow

Linear Programming

A Hotdog

A combination of A combination of pork, grain, and pork, grain, and sawdust, …sawdust, …

Constraints: Constraints: •Amount of Amount of moisturemoisture•Amount of Amount of protean,protean,•……

The Hotdog Problem

Given today’s prices,what is a fast algorithm to find the cheapest hotdog?

There are deep ideas within the simplicity.

=

AbstractionAbstraction

Goal: Understand and think about complex things in simple ways.

There are deep ideas within the simplicity. Rudich www.discretemath.com

Abstract Out Essential Details

Cost: 29, 8, 1, 2

Amount to add: x1, x2, x3, x4

pork

grai

nw

ater

saw

dust

3x1 + 4x2 – 7x3 + 8x4 122x1 - 8x2 + 4x3 - 3x4 24

-8x1 + 2x2 – 3x3 - 9x4 8x1 + 2x2 + 9x3 - 3x4 31

Constraints: Constraints: •moisturemoisture•protean,protean,•……

29x1 + 8x2 + 1x3 + 2x4Cost of Hotdog:Cost of Hotdog:

3x1 + 4x2 – 7x3 + 8x4 122x1 - 8x2 + 4x3 - 3x4 24

-8x1 + 2x2 – 3x3 - 9x4 8x1 + 2x2 + 9x3 - 3x4 31

29x1 + 8x2 + 1x3 + 2x4

Subject to:Subject to:

Minimize:Minimize:

Abstract Out Essential Details

A Fast Algorithm

For decades people thought that there was no fast algorithm.

Then one was found!

Theoretical Computer Sciencefinds new algorithms every day.

3x1 + 4x2 – 7x3 + 8x4 12

2x1 - 8x2 + 4x3 - 3x4 24

-8x1 + 2x2 – 3x3 - 9x4 8x1 + 2x2 + 9x3 - 3x4 31

29x1 + 8x2 + 1x3 + 2x4

Subject to:Subject to:

Minimize:Minimize:

End