Algorithm Design and Analysis - Penn State College of ...ads22/courses/cse565/F10/www/lec-notes/CSE565-F10-Lec... · 8/30/10 A. Smith; based on slides by E. Demaine, C. Leiserson,

8/30/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Adam Smith

Algorithm Design and Analysis

LECTURE 2 Analysis of Algorithms

• Stable matching problem

• Asymptotic growth

Stable Matching Problem

• Unstable pair: man m and woman w are unstable if

– m prefers w to his assigned match, and

– w prefers m to her assigned match

• Unstable pairs have an incentive to elope

• Stable matching: no unstable pairs.


Zeus Amy Clare Bertha

Yancey Bertha Clare Amy

Xavier Amy Clare Bertha

1st 2nd 3rd

Men’s Preference Profile

favorite least favorite

Clare Xavier Zeus Yancey

Bertha Xavier Zeus Yancey

Amy Yancey Zeus Xavier

1st 2nd 3rd

Women’s Preference Profile


Stable Matching Problem

• Input: preference lists of n men and n women

• Goal: find a stable matching if one exists


Zeus Amy Clare Bertha

Yancey Bertha Clare Amy

Xavier Amy Clare Bertha

1st 2nd 3rd



Clare Xavier Zeus Yancey

Bertha Xavier Zeus Yancey

Amy Yancey Zeus Xavier

1st 2nd 3rd




Review Questions

• In terms of n, what is the length of the input to

the Stable Matching problem, i.e., the number of

entries in the tables?

• How many bits do they take to store?

(Answer: 2n2 list entries, or 2n2log n bits)


Review Questions

• Brute force algorithm: an algorithm that

checks every possible solution.

• In terms of n, what is the running time for the

brute force algorithm for checking whether a

give matching is stable?

• In terms of n, what is the running time for the

brute force algorithm for Stable Matching

Problem? (Assume your algorithm goes over all

possible perfect matchings.)

(Answer: n! (time to check if a matching is stable) = (n! n2) )

Propose-and-Reject Algorithm

• Propose-and-reject algorithm. [Gale-Shapley 1962]


Initialize each person to be free.

while (some man is free and hasn't proposed to every woman) {

Choose such a man m

w = 1st woman on m's list to whom m has not yet proposed

if (w is free)

assign m and w to be engaged

else if (w prefers m to her fiancé m')

assign m and w to be engaged, and m' to be free

else

w rejects m

}

Proof of correctness

Three claims: The algorithm always

1. terminates,

2. matches everyone (matching is “perfect”), and

3. outputs a stable matching


Zeus Bertha Amy Diane Erika Clare

Yancey Amy Clare Diane Bertha Erika

Xavier Bertha Clare Erika Diane Amy

Wyatt Diane Amy Bertha Clare Erika

Victor Bertha Diane Amy Erika Clare

0th 1st 2nd 3rd 4th


Erika Yancey Zeus Wyatt Xavier Victor

Diane Victor Yancey Zeus Xavier Wyatt

Clare Wyatt Yancey Xavier Zeus Victor

Bertha Xavier Yancey Wyatt Victor Zeus

Amy Zeus Wyatt Victor Yancey Xavier

0th 1st 2nd 3rd 4th


Bertha

Xavier

Amy

Victor

Clare

Wyatt

Diane

Zeus

Erika

Yancey

STOP

- Everyone matched. - Stable matching!

Victor

Bertha

Diane

Wyatt

Amy

Yancey

Diane

Yancey

Bertha

Wyatt

Amy

Wyatt

Bertha

Zeus

Clare

Yancey

Bertha

Yancey

Proof of Correctness: Termination

• Claim. Algorithm terminates after at most n2

iterations of while loop.

• Pf. Each time through the loop a man proposes

to a new woman. There are only n2 possible

proposals.


Wyatt

Victor

1st

A

B

2nd

C

D

3rd

C

B

A Zeus

Yancey

Xavier C

D

A

B

B

A

D

C

4th

E

E

5th

A

D

E

E

D

C

B

E

Bertha

Amy

1st

W

X

2nd

Y

Z

3rd

Y

X

V Erika

Diane

Clare Y

Z

V

W

W

V

Z

X

4th

V

W

5th

V

Z

X

Y

Y

X

W

Z

An instance where n(n-1) + 1 proposals required

Propose-and-Reject Algorithm

• Observation 1. Men propose to women in

decreasing order of preference.

• Observation 2. Once a woman is matched, she

never becomes unmatched; she only "trades up."


Zeus Bertha Amy Diane Erika Clare

Yancey Amy Clare Diane Bertha Erika

Xavier Bertha Clare Erika Diane Amy

Wyatt Diane Amy Bertha Clare Erika

Victor Bertha Diane Amy Erika Clare

0th 1st 2nd 3rd 4th


Bertha

Amy

Clare

Erika

Bertha

Diane

Amy Diane

Bertha Amy

Bertha

Clare Bertha

Erika Yancey Zeus Wyatt Xavier Victor

Diane Victor Yancey Zeus Xavier Wyatt

Clare Wyatt Yancey Xavier Zeus Victor

Bertha Xavier Yancey Wyatt Victor Zeus

Amy Zeus Wyatt Victor Yancey Xavier

0th 1st 2nd 3rd 4th


Xavier

Victor

Wyatt

Yancey

Victor

Wyatt

Yancey

Yancey

Wyatt

Wyatt

Zeus

Yancey

Yancey

Proof of Correctness: Perfection

• Claim. All men and women get matched.

• Proof: (by contradiction)

– Suppose, for sake of contradiction, some guy, say

Zeus, is not matched upon termination of algorithm.

– Then some woman, say Amy, is not matched upon

termination.

– By Observation 2, Amy was never proposed to.

– But Zeus proposes to everyone, since he ends up

unmatched.


Proof of Correctness: Stability

•Claim. No unstable pairs.

•Proof: (by contradiction)

– Suppose A-Z is an unstable pair: they prefer each other to their

partners in Gale-Shapley matching S*.

– Case 1: Z never proposed to A.

Z prefers his GS partner to A.

A-Z is stable.

– Case 2: Z proposed to A.

A rejected Z (right away or later)

A prefers her GS partner to Z.

A-Z is stable.

– In either case A-Z is stable, a contradiction.


men propose in decreasing order of preference

women only trade up

Stable Roommate Problem

• Stable roommate problem

– 2n people; each person ranks others from 1 to 2n-1.

– Assign roommate pairs so that no unstable pairs.

• Exercise. Where does the correctness proof break

down for the roommates version?


B

Bob

Chris

Adam C

A

B

D

D

Doofus A B C

D

C

A

1st 2nd 3rd

A-B, C-D B-C unstable A-C, B-D A-B unstable A-D, B-C A-C unstable

Efficient Implementation

• We describe O(n2) time implementation.

• Assume men have IDs 1,…, n, and so do women.

• Engagements data structures:

– a list of free men, e.g., a queue.

– two arrays wife[m], and husband[w].

• set entry to 0 if unmatched

• if m matched to w then wife[m]=w and husband[w]=m

• Men proposing data structures:

– an array men-pref[m,i]= ith women on mth list

– an array count[m] = how many proposals m made.

• In Python: http://euler.slu.edu/~goldwasser/courses/slu/csci314/2006_Spring/lectures/marriageAlgorithm/


Efficient Implementation

• Women rejecting/accepting data structures

– Does woman w prefer man m to man m'?

– For each woman, create inverse of preference list of men.

– Constant time queries after O(n) preprocessing per woman.


for i = 1 to n

inverse[pref[i]] = i

Pref

1st

8

2nd

7

3rd

3

4th

4

5th

1 5 2 6

6th 7th 8th

Inverse 4th 2nd 8th 6th 5th 7th 1st 3rd

1 2 3 4 5 6 7 8

Amy

Amy

Amy prefers man 3 to 6 since inverse[3] < inverse[6]

2 7

Summary

• Stable matching problem. Given n men and n

women, and their preferences, find a stable

matching if one exists.

• Gale-Shapley algorithm. Guarantees to find a

stable matching for every problem instance.

– (Also proves that stable matching always exists)

• Time and space complexity:

O(n2), linear in the input size.


Brief Syllabus

• Reminders

– Worst-case analysis

– Asymptotic notation

– Basic Data Structures

• Design Paradigms

– Greedy algorithms, Divide and conquer, Dynamic

programming, Network flow and linear programming,

randomization

• Analyzing algorithms in other models

– Parallel algorithms, Memory hierarchies (?)

• P, NP and NP-completeness


Measuring Running Time

• Focus on scalability: parameterize the running

time by some measure of “size”

– (e.g. n = number of men and women)

• Kinds of analysis

– Worst-case

– Average-case (requires knowing the distribution)

– Best-case (how meaningful?)

• Exact times depend on computer; instead

measure asymptotic growth



Asymptotic notation

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all n n0.

O-notation (upper bounds):

EXAMPLE: 2n2 = O(n3)

functions, not values

(c = 1, n0 = 2)

funny, “one-way” equality


Set Definition

O(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all n n0 }

EXAMPLE: 2n2 O(n3)

(Logicians: n.2n2 O( n.n3), but it’s convenient to be sloppy, as long as we understand what’s really going on.)

Examples

• 106 n3+ 2n2 -n +10 = O(n3)

• n + log(n) = O(n )

• n (log(n) + n ) = O(n3/2)

• n = O(n2)



-notation (lower bounds)

(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 cg(n) f(n) for all n n0 }

EXAMPLE: (c = 1, n0 = 16)

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).


-notation (lower bounds)

• Be careful: “Any comparison-based sorting algorithm

requires at least O(n log n) comparisons.”

– Meaningless!

– Use for lower bounds.


-notation (tight bounds)

(g(n)) = O(g(n)) (g(n))

EXAMPLE:

Polynomials are simple:

ad nd + ad–1n

d–1 + + a1n + a0 = (nd)


o-notation and -notation

o(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 f(n) < cg(n) for all n n0 }

EXAMPLE: (n0 = 2/c)

O-notation and -notation are like and .

o-notation and -notation are like < and >.

2n2 = o(n3)


o-notation and -notation

(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 cg(n) < f(n) for all n n0 }

EXAMPLE: (n0 = 1+1/c)

O-notation and -notation are like and .

o-notation and -notation are like < and >.


Common Functions: Asymptotic Bounds

• Polynomials. a0 + a1n + … + adnd is (nd) if ad > 0.

• Polynomial time. Running time is O(nd) for some

constant d independent of the input size n.

• Logarithms. log a n = (log b n) for all constants a, b > 0.

For every x > 0, log n = O(nx).

• Exponentials. For all r >1 and all d > 0, nd = O(rn).

• Factorial.

grows faster than every exponential

can avoid specifying the base log grows slower than every polynomial

Every exponential grows faster than every polynomial

n! = (√

2πn)(n

e

)n

(1 + o(1)) = 2Θ(n log n)

8/30/10

A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Sort by asymptotic order of growth

a) n log(n)

b)

c) log(n)

d) n2

e) 2n

f) n

g) n!

h) n1,000,000

i) n1/log(n)

j) log(n!)

k)

l) (

n

n/2

)(

n

2

)

Documents

Algorithm Design and Analysis - Penn State College of ...ads22/courses/cse565/F10/www/lec-notes/CSE565-F10-Lec... · 8/30/10 A. Smith; based on slides by E. Demaine, C. Leiserson,