12
Data Structure Chapter 1 & 3 Dr. Patrick Chan School of Computer Science and Engineering South China University of Technology Introduction and Algorithm Analysis 2 Lec 1 & 3: Introduction and Algorithm Analysis Outline Introduction (Ch1) Philosophy of Data Structure (1.1) Abstract Data Types and Data Structures (1.2) Problems, Algorithms and Programs (1.3) 3 Lec 1 & 3: Introduction and Algorithm Analysis Outline Algorithm Analysis (Ch3) Best, Worst and Average Cases (3.2) Fast Computer or Fast Algorithm? (3.3) Asymptotic Analysis (3.4) Multiple Parameters (3.8) Space Bounds (3.9) 4 Lec 1 & 3: Introduction and Algorithm Analysis Introduction: Efficient Programs? Program is the soul of a machine The objective of learning “data structure” is to improve the efficiency of a program Shorter running time Less memory

Introduction: Efficient Programs?

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Data Structure

Chapter 1 &

3

Dr.

Patr

ick C

han

Scho

ol

of

Com

pute

r S

cie

nce a

nd E

ngin

eeri

ng

South

Chin

a U

niv

ers

ity o

f Technolo

gy

Introduction and

Algorithm Analysis

2

Lec1 & 3: Introduction and Algorithm Analysis

Outline

In

tro

du

ction

(C

h1

)

P

hilo

soph

y o

f D

ata

Str

uctu

re (

1.1

)

A

bstr

act

Data

Types a

nd D

ata

Str

uctu

res (

1.2

)

P

roble

ms,

Alg

orith

ms

and P

rog

ram

s (

1.3

)

3

Lec1 & 3: Introduction and Algorithm Analysis

Outline

A

lgori

thm

An

aly

sis

(C

h3

)

B

est,

Wors

t and A

vera

ge C

ases (

3.2

)

F

ast

Com

pute

r or

Fast

Alg

orith

m? (

3.3

)

A

sym

pto

tic A

naly

sis

(3.4

)

M

ultip

le P

ara

mete

rs (

3.8

)

S

pace B

ounds (

3.9

)

4

Lec1 & 3: Introduction and Algorithm Analysis

Introduction: Efficient Programs?

P

rog

ram

is t

he

so

ul

of

a m

ach

ine

T

he o

bje

ctive

of

lea

rnin

g “

da

ta s

tru

ctu

re”

is

to im

pro

ve t

he e

ffic

ien

cy o

f a

pro

gra

m

S

hort

er

runnin

g t

ime

Less m

em

ory

5

Lec1 & 3: Introduction and Algorithm Analysis

Introduction: Efficient Programs?

1

98

2:

Inte

l 8

02

86

16 M

Hz

(16,0

00,0

00 H

z)

2

01

2:

Inte

l C

ore

i7

Ext

rem

e

3.8

0 G

Hz (

3,8

00,0

00,0

00H

z)

2

37

.5 t

ime

s d

iffe

ren

ce

For

exa

mple

, a p

rogra

m

C

ore

i7 E

xtre

me n

eeds 1

sec

80286 n

eeds 3

.95 m

ins

N

ow

ad

ays,

the h

ard

ware

is v

ery

po

werf

ul.

Wh

y d

o w

e s

till

need

to w

rite

an

effic

ien

t p

rog

ram

?

6

Lec1 & 3: Introduction and Algorithm Analysis

Introduction: Efficient Programs?

M

ore

po

we

rfu

l

co

mp

ute

rsen

co

ura

ge

mo

re c

om

ple

x

ap

pli

ca

tio

ns

M

ore

calc

ula

tions

R

esourc

es d

em

andin

g

A

mo

re e

ffic

ien

t p

rog

ram

is a

lwa

ys

de

sir

ab

le

Wit

h g

rea

t p

ow

er

co

me

s g

rea

t re

sp

on

sib

ilit

y!

7

Lec1 & 3: Introduction and Algorithm Analysis

What is D

ata Structure?

P

rop

er

da

ta s

tru

ctu

recan

ma

ke s

ign

ific

an

t

diffe

ren

ce

in a

pro

gra

m r

un

nin

g t

ime

A

eff

icie

nt

solu

tio

nm

ay b

e o

bta

ined

by

se

lectin

g p

rop

er

data

str

uctu

res

D

ata

Str

uc

ture

is a

ny d

ata

re

pre

se

nta

tion

an

d i

ts a

ssocia

ted o

pera

tion

s

e.g

. In

teger:

S

um

mation

Str

ing:

R

epla

ce

8

Lec1 & 3: Introduction and Algorithm Analysis

Data Structure Philosophy

R

ea

l N

um

ber

is b

etter

than

In

tege

r?

E

ve

ry d

ata

str

uctu

re h

as c

osts

an

d b

en

efi

ts

N

o d

ata

str

uctu

re i

s b

ett

er

than a

noth

er

in a

ll

situ

ations

A

da

ta s

tru

ctu

re r

equ

ires:

S

pac

efo

r each d

ata

ite

m i

t sto

res

T

ime

to p

erf

orm

each b

asic

opera

tion

P

rog

ram

min

g e

ffo

rt

No

9

Lec1 & 3: Introduction and Algorithm Analysis

Data Structure Philosophy

E

ach

pro

ble

m h

as c

on

str

ain

ts o

n a

vailab

le t

ime

an

d

sp

ac

e

O

nly

after

a c

are

ful analy

sis

of

pro

ble

m c

hara

cte

ristics

can w

e k

now

the b

est

data

str

uctu

re for

the t

ask

D

ata

Str

uc

ture

Se

lec

tio

n:

A

naly

ze

the p

roble

m a

nd d

ete

rmin

e t

he r

esourc

e

constr

ain

ts

D

ete

rmin

eth

e b

asic

opera

tion

s a

nd q

uantify

the

resourc

e c

onstr

ain

ts f

or

each o

pera

tion

S

ele

ct

the d

ata

str

uctu

re t

hat

best

meets

these

req

uirem

ent

10

Lec1 & 3: Introduction and Algorithm Analysis

Definitions

T

yp

e:

a c

olle

ction o

f valu

es

e.g

.,Boolean type

: colle

ction o

f tr

ue o

r fa

lse v

alu

eInteger type

: colle

ction o

f 1,2

,3…

va

lue

D

ata

ty

pe

:a t

yp

ean

d a

co

llection

of

op

era

tion

sth

at

man

ipu

late

th

e t

yp

e

e.g

.,Integer data type

:

Inte

ger

type a

nd +

-x ÷

ope

ratio

ns

D

ata

Ite

m:

a p

iece o

f in

form

ation

of

a r

eco

rd d

raw

n

from

a d

ata

typ

e

e.g

.,true

or false

: a p

iece o

f in

form

ation f

rom

a B

oole

an t

ype

A

data

ite

mis

said

to b

e a

mem

ber

of

a d

ata

type

11

Lec1 & 3: Introduction and Algorithm Analysis

Definitions

A

tom

ic D

ata

Ty

pe

:co

nta

in n

o s

ub

part

s

e.g

. In

teg

er,

Boole

an

S

tru

ctu

re D

ata

Ty

pe

:co

nta

in s

eve

ralp

iece

s

of

info

rma

tio

n

e.g

. A

rra

y,

Str

ing

12

Lec1 & 3: Introduction and Algorithm Analysis

Definitions

A

bs

tra

ct

Da

ta T

yp

e:

a d

efin

itio

n for

a d

ata

typ

e

so

lely

in

term

s o

fa s

et

of va

lue

san

d a

set

of

op

era

tio

ns

on

th

at

data

typ

e

A

DT

opera

tion is d

efined b

y its

inputs

and o

utp

uts

H

ide im

ple

menta

tion d

eta

ils (

Encapsula

tion)

D

ata

Str

uc

ture

:th

e p

hysic

al

imp

lem

en

tation

of

an

AD

T

O

pera

tions a

ssocia

ted w

ith t

he A

DT

are

im

ple

mente

d

by s

ubro

utines

(functions)

U

sually

refe

rs t

o a

n o

rganiz

ation f

or

data

in m

ain

me

mory

13

Lec1 & 3: Introduction and Algorithm Analysis

Definitions

F

ile

Str

uc

ture

:a

n o

rga

niz

atio

n f

or

data

on

pe

rip

hera

l sto

rag

e

e.g

. D

isk d

rive

14

Lec1 & 3: Introduction and Algorithm Analysis

Logical vsPhysical Form

D

ata

ite

ms

ha

ve b

oth

a l

ogic

al

an

d a

ph

ysic

al

form

L

og

ical

Fo

rm

D

efin

itio

n o

f th

e d

ata

ite

m w

ith

in a

n A

DT

e.g

. In

teg

ers

in

math

em

atica

l se

nse

: +

, -

P

hys

ical

Fo

rm

Im

ple

me

nta

tio

n o

f th

e d

ata

ite

m w

ith

in a

da

ta

str

uctu

re

e.g

. 1

6/3

2 b

it i

nte

gers

: ove

rflo

w

15

Lec1 & 3: Introduction and Algorithm Analysis

Logical vsPhysical Form

Da

ta T

yp

e

AD

T:

Type

Op

era

tion

s

Data

Ite

ms:

Physic

al

Form

Data

Str

uctu

re:

Sto

rage

Spa

ce

Su

bro

utine

s

Data

Ite

ms:

Logic

al

Form

16

Lec1 & 3: Introduction and Algorithm Analysis

Problem,

Algorithm and Program

P

rob

lem

:A

task t

o b

e

pe

rfo

rme

d

B

est

thought

of

as in

puts

and m

atc

hin

g o

utp

uts

P

rob

lem

definitio

n s

hould

inclu

de c

onstr

ain

ts o

n t

he

resourc

es

that

ma

y b

e

consum

ed

by a

ny

accepta

ble

solu

tio

n

17

Lec1 & 3: Introduction and Algorithm Analysis

Problem,

Algorithm and Program

A

lgo

rith

m:

a m

eth

od t

o

so

lve a

pro

ble

m

C

orr

ect

N

o a

mbig

uity

A

series o

f concre

te s

teps

A

fin

ite n

um

ber

of

ste

ps

T

erm

inate

18

Lec1 & 3: Introduction and Algorithm Analysis

Problem,

Algorithm and Program

P

rog

ram

:a

n in

sta

nce

for

an

alg

ori

thm

in s

om

e

pro

gra

mm

ing

la

ng

ua

ges

19

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Evaluation

M

an

y a

ppro

ach

es (

alg

orith

ms)

to s

olv

e a

pro

ble

m.

Wh

ich

on

e i

s t

he b

est?

T

wo

cri

teri

a:

E

ffic

ien

cy

C

on

ce

rn o

f D

ata

Str

uctu

res a

nd

Alg

orith

m A

na

lysis

E

as

y t

o u

nd

ers

tan

d

C

on

ce

rn o

f S

oft

wa

re E

ng

ine

erin

g

T

he

y a

re c

on

flic

tin

g

20

Lec1 & 3: Introduction and Algorithm Analysis

Min

nie

Pig

let

Sli

nky

Pin

occh

ioS

qu

irt

Su

lly

Mic

key

Mik

e

Algorithm Evaluation

E

xam

ple

: H

ow

ca

n w

e f

ind S

qu

irt?

Lo

w E

ffic

ien

cy

Hig

h E

ffic

ien

cy

Ea

sy

Dif

ficu

lt

Se

qu

en

tia

l S

ea

rch

Bin

ary

Se

arc

h

21

Lec1 & 3: Introduction and Algorithm Analysis

How to Compare Efficiency?

T

wo

Me

tho

ds:

E

mp

iric

al

Co

mp

ari

so

n

R

un

a p

rog

ram

an

d s

ee

how

mu

ch

of

the r

eso

urc

es

is u

sed

M

ach

ine d

ep

en

de

nt

D

iffic

ult t

o s

etu

pa

nd

may

be

b

iase

d

A

lgo

rith

m A

naly

sis

E

stim

ate

th

e t

ime

and

sp

ace

need

ed

for

a p

rog

ram

M

ach

ine in

de

pe

nd

en

t

22

Lec1 & 3: Introduction and Algorithm Analysis

Algorith

m A

nalysis

Running Tim

e T(n)

C

ritical

resou

rce o

f a p

rogra

m i

s m

ost

oft

en

its

Ru

nn

ing

Tim

e

W

hic

h o

ne h

as t

he l

on

gest

run

nin

g t

ime?

A

ssu

me n

is input

by u

se

rs

a = n;

b = n*n;

c = n*n*n;

d = n*n*n*n;

ans= a+b+c+d;

ans

= 0;

for i = 1 to 3

ans= ans+n;

end

ans

= 0;

for i = 1 to n

ans= ans+i;

end

Ru

nn

ing

tim

e d

oe

s n

ot re

lyo

n u

se

r’s

in

pu

tR

un

nin

g t

ime r

elie

s o

n

us

er’

s in

pu

t

23

Lec1 & 3: Introduction and Algorithm Analysis

sum = 0;

for i = 1 to n

for j = 1 to n

sum = sum+ 1;

end

end

Algorith

m A

nalysis

Running Tim

e T(n)

F

ocu

s o

n t

he p

rog

ram

wh

ich

dep

en

ds o

n “

siz

e”

of

inp

uts

T

(n)

for

som

e f

un

cti

on

Ton

in

pu

t s

ize

n

sum = 0;

c:

the

ru

nn

ing

tim

e t

o a

ssig

n a

va

lue

to

a v

ari

ab

le

T(n) =c

c:

the

ru

nn

ing

tim

e t

o a

ssig

n

a v

alu

e t

o a

vari

ab

le

T(n) =c +cn2

This c is not really important

(not relate to n)

Lo

op

n

tim

es

Lo

op

n

tim

es

24

Lec1 & 3: Introduction and Algorithm Analysis

s1 = 1;

s2 = 1;

s3 = 1;

for a = 1 to n/2

s1 = s1+ s1;

end

for b = 1 to n*n

s2 = s2 * s2;

end

for i = 1 to n

for j = 1 to log(n)

s3 = s1+s2+s3;

end

end

Algorith

m A

nalysis

Running Tim

e T(n): Exercise

c cn/2

cn2

cnlog n

T(n) =c +cn/2

+ cn2 + c n logn

25

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis

Growth Rate

X-a

xis

: in

pu

t s

ize

Y-a

xis

: a

ny

mea

su

re o

f c

os

t

(e.g

. ti

me

or

sp

ac

e)

26

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis

Best, W

orst, Average Cases

S

am

e i

np

ut

siz

e m

ay r

eq

uir

e d

iffe

ren

t

am

ou

nts

of

run

nin

g t

ime

F

or

exam

ple

:

Sequen

tial

searc

hfo

r K

in a

n a

rra

y o

f n

inte

gers

B

eg

in a

t fir

st

ele

me

nt

in a

rra

y a

nd

loo

k a

t ea

ch

ele

me

nt in

tu

rn u

ntil K

is f

ou

nd

B

est

Ca

se

:

W

ors

t C

ase

:

A

ve

rag

e C

ase

:

1 n

(n+

1)/

2

Min

nie

Pig

let

Slin

ky

Pin

oc

ch

ioS

qu

irt

Su

lly

Mic

ke

yM

ike

27

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis

Best, W

orst, Average Cases

W

hic

h m

easu

re s

ho

uld

be u

se

d?

B

est

ca

se

M

ay h

ap

pen

rare

ly

T

oo o

ptim

istic

W

ors

t case

U

pp

er

bo

un

d

Im

port

an

t to

real tim

e a

lgori

thm

s

A

vera

ge

case

T

he f

air

est

measu

re

D

iffic

ult t

o d

ete

rmin

e

Need t

o k

no

w t

he a

ll po

ssib

le i

nputs

and t

heir c

osts

28

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm A

nalysis: Running Tim

e T(n)

Fast Computer or Fast Algorithm?

If

we w

an

t to

red

uce t

he r

un

nin

g t

ime

of

a

pro

gra

m,

wh

at

sh

ou

ld w

e d

o?

B

uy a

faste

r co

mp

ute

r?

W

rite

a f

aste

r alg

ori

thm

?

29

Lec1 & 3: Introduction and Algorithm Analysis

nnew/ nold

Ch

an

ge

nnew

nold

T(n)

Algorithm A

nalysis: Running Tim

e T(n)

Fast Computer or Fast Algorithm?

O

ld C

om

pu

ter

(10

,00

0 c

od

e/h

ou

r)

N

ew

Co

mp

ute

r(1

00

,00

0 c

od

e/h

ou

r)

Siz

e o

f in

put

that

ca

n b

e p

roce

sse

d

us

ing O

LD

Co

mp

ute

rin

one h

our

Siz

e o

f in

put

that

ca

n b

e p

roce

sse

d

us

ing N

EW

Co

mp

ute

rin

one h

our

10n

20n

2n2

2n

5n log n

1,000

500

70

13

250

10,000

5,000

223

16

1,842

nnew= 10nold

nnew= 10nold

nnew= √10nold

nnew= nold+ 3

√10 nold< nnew< 10 nold

10

10

3.16

~1

7.37

30

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis

W

e w

ou

ld l

ike

to

kn

ow

wh

at

the

ch

an

ge

of

the

co

mp

lex

ity

is w

he

n n

gro

ws t

o ∞

T

hre

e d

iffe

ren

t m

ea

su

res:

B

ig-O

h (

O)

B

ig-O

meg

a (Ω ΩΩΩ

)

B

ig-T

heta

(Θ ΘΘΘ

)

31

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Oh

In

dic

ate

s t

he

up

pe

r b

ou

nd

of

a g

row

th r

ate

D

efi

nit

ion

For T(n)

a n

on-n

ega

tively

valu

ed

fu

nction

,

T(n)

is i

n t

he

set O(f(n))

if t

here

exis

ttw

o p

ositiv

e c

on

sta

nts

ca

nd

n0

su

ch

th

at T(n)

≤c f (n)

for

all n> n0

n0

is t

he s

malle

st

valu

e o

f n

for

whic

h t

he c

laim

of an

upper

bound h

old

s t

rue

A

ctu

ally

valu

e o

f c

is irr

ele

vant

32

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Oh

f (n) = n2

c= 3

n0= 1

T(n) = 3n2

T(n)

≤cf(n)

3n2

≤cf(n)

if T(n)

≤c f (n)

for

all n> n0,

T(n)

is i

n t

he s

et O(f(n))

T(n) = 3n2 + n

T(n)

≤cf(n)

3n2 + n

≤3n2 + n2

=4n2

f (n) = n2

c= 4

n0= 1

T(n)

is i

n O(n2)

=3n2

By

su

bsti

tuti

ng

,B

y s

ub

sti

tuti

ng

,

3n2

≤4n2

3n2 + n

T(n)

is i

n O(n2)

33

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm A

nalysis: Big-Oh

Exercise

T(n) =c +cn/2+ cn2 + cnlogn

if T(n)

≤c f (n)

for

all n> n0,

T(n)

is i

n t

he s

et O(f(n))

T(n)

≤sf(n)

≤cn

2+cn

2/2+ cn2+ cn2

=(c+ c/2 + c+ c)n2

f (n) = n2

s= 7c/2

n0= 1

By

su

bsti

tuti

ng

,

T(n)

is i

n O(n2)

c +cn/2+ cn2+ cnlogn

=( 7c/2 )n2

n >

lo

g n

, a

s n

→∞ ∞∞∞

c +cn/2+ cn2+ cnlogn

≤( 7c/2 )n2

34

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Oh

G

ive

n T(n) = 3n

W

e k

no

w t

ha

t “T(n) = 3n

is i

n O(n)”

C

an

we s

ay “T(n) = 3n

is i

n O(n3)”

?

Y

es

bu

t th

e t

igh

tes

t u

ppe

r bo

un

dis

pre

ferr

ed

35

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Oh

B

ig-O

h V

S W

ors

t C

as

e

B

ig-O

hre

fers

to a

gro

wth

rate

W

ors

t case

refe

rs t

o t

he w

ors

t in

put

fro

m

am

ong t

he c

hoic

es f

or

possib

le i

nputs

of

a g

iven

siz

e

e.g

. S

eque

ntial

Searc

h

B

ig-o

h: T(n)

is in

O(n)

W

ors

t C

ase: n

36

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Oh

O(1):

co

nsta

nt

O(log2n):

lo

gari

thm

ic

O(log 22n):

lo

g s

quare

d

O(n):

linea

r

O(nlog2n):

n log

n

O(n2):

q

uad

ratic

O(n3):

cub

ic

O(2n):

exp

on

en

tia

l

37

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Oh

B

ig-O

h V

S W

ors

t C

as

e

B

ig-O

hre

fers

to a

gro

wth

rate

W

ors

t case r

efe

rs t

o t

he w

ors

t in

put

fro

m a

mong

the c

hoic

es f

or

possib

le i

np

uts

of

a g

ive

n s

ize

e.g

. S

eque

ntial

Searc

h

B

ig-o

h: T(n)

is in

O(n)

W

ors

t C

ase: n

38

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Omega

In

dic

ate

s t

he

lo

we

r b

ou

nd

of

a g

row

th r

ate

D

efi

nit

ion

For T(n)

a n

on-n

ega

tively

valu

ed

fu

nction

,

T(n)

is i

n t

he

set Ω(g(n))

if t

here

exis

ttw

o p

ositiv

e c

on

sta

nts

ca

nd

n0

su

ch

th

at T(n)

≥c g(n)

for

all n> n0

n0

is t

he s

malle

st

valu

e o

f n

for

whic

h t

he c

laim

of

an

upper

bound h

old

s t

rue

T

he a

ctu

ally v

alu

e o

f c

is irr

ele

vant

39

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Omega

f (n) = n2

c= 3

n0= 1

T(n) = 3n2

T(n)

≥cf(n)

3n2

≥cf(n)

if T(n)

≥c f (n)

for

all n> n0,

T(n)

is i

n t

he s

et Ω(f (n))

T(n) = 3n2 + n

T(n)

≥cf(n)

3n2 + n

≥3n2

=3n2

f (n) = n2

c= 3

n0= 1

T(n)

is i

n Ω ΩΩΩ(n2)

=3n2

By

su

bsti

tuti

ng

,B

y s

ub

sti

tuti

ng

,

3n2

≥3n2

3n2 + n

T(n)

is i

n Ω ΩΩΩ(n2)

40

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Theta

W

he

n O

an

d Ω

are

th

e s

am

e,

we

in

dic

ate

th

is

situ

atio

n b

y u

sin

g Θ

no

tatio

n

D

efi

nit

ion

An

alg

orith

m i

s s

aid

to b

e Θ(h(n))

if it

is i

n O(h(n))

an

d Ω(h(n))

41

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Big-Theta

E

xa

mp

le 1

T(n) = 1

⇒ ⇒⇒⇒O(1)

andΩ(1) ⇒ ⇒⇒⇒

Θ(1)

E

xa

mp

le 2

:

T(n) = c1n + c2⇒ ⇒⇒⇒

Ο(n)

andΩ(n) ⇒ ⇒⇒⇒

Θ(n)

sum = 0;

for (i=1; i<=n; i++)

sum += n;

a = b;

42

Lec1 & 3: Introduction and Algorithm Analysis

Small Exercise

W

hat

is t

he B

ig-O

, B

ig-O

meg

a a

nd

Big

-Th

eta

of

the

follo

win

g p

rog

ram

?

T(n)

O(n2),Ω(n2),Θ(n2)

sum = 0;

for (i=1; i<=n; i++)

//first loop

for (j=1; j<=i; j++) //double loop

sum++;

for (k=0; k<n; k++)

//second loop

A[k] = k;

nc

ic

cn i

3

1

21

++

=∑ =

nc

nn

cc

32

12

)1(

++

+=

43

Lec1 & 3: Introduction and Algorithm Analysis

Small Exercise

W

hat

is t

he B

ig-O

, B

ig-O

meg

a a

nd

Big

-Th

eta

of

the

fo

llow

ing

pro

gra

m?

T(n) = c1+ c2n2+ c1+ c2n(n+1)/2

Ω(n2),O(n2),Θ(n2)

sum1 = 0;

for(i=1;i<=n;i++)

//first double loop

for(j=1;j<=n;j++) //do n times

sum1++;

sum2 = 0;

for(i=1;i<=n;i++)

//second double loop

for(j=1;j<=i;j++) //do i times

sum2++;

44

Lec1 & 3: Introduction and Algorithm Analysis

l

Algorithm Analysis: Case Study

Binary Search

// Return position of element in sorted

// array of size n with value K.

intbinary(int

array[], intn, intK)

int

l = -1;

int

r = n;

// l, r are beyond array bounds

while (l+1 != r)

// Stop when l, r meet

inti = (l+r)/2;

// Check middle

if (K < array[i]) r = i;

// Left half

if (K == array[i]) return i;

// Found it

if (K > array[i]) l = i;

// Right half

return n; // Search value not in array

H

ow

man

y e

lem

en

ts a

re e

xam

ined

in

wo

rst

case?

11

13

21

26

29

36

40

41

45

51

54

56

65

72

77

83

01

23

45

67

89

10

11

12

13

14

15

Po

sit

ion

Va

lue

lr

iir

iri

Find: 45

45

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis: Case Study

Binary Search

H

ow

man

y e

lem

en

ts a

re e

xam

ined

in

wo

rst

case?

T(n) = T(n/2) + 1

wh

ere

n>1

an

d T(1) = 1

T

here

fore

, T(n) = log2n + 1

C

ost

is Θ(log2n)

11

13

21

26

29

36

40

41

45

51

54

56

65

72

77

83

01

23

45

67

89

10

11

12

13

14

15

Po

sit

ion

Va

lue

46

Lec1 & 3: Introduction and Algorithm Analysis

Algorithm Analysis

Other Control Statements

W

hil

e l

oo

p

An

aly

ze l

ike a

for

loo

p

If

sta

tem

en

t

Ta

ke g

rea

ter

co

mple

xity o

f th

en

/els

e c

lau

se

s

S

wit

ch

sta

tem

en

t

Ta

ke c

om

ple

xity o

f m

ost

expe

nsiv

e c

ase

S

ub

rou

tin

e c

all

Co

mp

lexity o

f th

e s

ubro

utin

e

47

Lec1 & 3: Introduction and Algorithm Analysis

Analyzing Problems

Multiple Parameters

C

om

pu

te t

he r

an

k o

rderin

g f

or

all

Cp

ixe

l valu

es i

n a

pic

ture

of

Pp

ixe

ls.

T(P, C) = C + P + C2

O(P + C2)

an

dΩ(P + C2)

Θ

(P + C2)

for (i=0; i<C; i++)

// Initialize count

count[i] = 0;

for (i=0; i<P; i++)

// Look at all pixels

count[value(i)]++; // Increment count

bubbleSort(count);

// Sort pixel counts

C: 2

56

co

lors

(8

bit

s)

P:

50

x 4

0 p

ixe

ls

48

Lec1 & 3: Introduction and Algorithm Analysis

Analyzing Problems

Space/Tim

e Tradeoff Principle

O

ne c

an

ofte

n r

edu

ce t

ime i

f o

ne i

s w

illin

g t

o

sacri

fice s

pace,

or

vic

e v

ers

a,

e.g

.

E

ncodin

g o

r packin

g i

nfo

rmation

B

oole

an

Fla

gs

B

oo

lean t

ake

s o

ne b

it,

but

a b

yte

is t

he s

malle

st

sto

rag

e,

so

pack

8 B

oo

lean

s i

nto

1 b

yte

T

able

looku

p

F

acto

ria

ls

C

om

pute

once,

sto

re r

esults,

use m

any t

imes