61A Extra Lecture 4

61A Extra Lecture 4

Thursday, February 19

0100010101101110011000110110111101100100011010010110111001100111

0100010101101110011000110110111101100100011010010110111001100111(Encoding)

What’s the point?

3http://pixshark.com/confused-face-clip-art.htm

http://pixshark.com/confused-face-clip-art.htm

What’s the point?

• Why do we encode things?



What’s the point?


• You don’t speak binary



What’s the point?



• Computers don’t speak English



What’s the point?



• Computers don’t speak English



A First Attempt

4

A First Attempt

• Let’s use an encoding

4

A First Attempt

• Let’s use an encoding

4

Letter Binary Letter Binarya 0 n 1b 1 o 0c 0 p 1d 1 q 1e 1 r 0f 0 s 1g 0 t 0h 1 u 0i 1 v 1j 1 w 1k 0 x 1l 1 y 0

m 1 z 0

Analysis

5

Analysis

Pros

5

Analysis

Pros

• Encoding was easy

5

Analysis

Pros


• Took a very small amount of space

5

Analysis

Pros



Cons

5

Analysis

Pros



Cons

• Decoding it was impossible

5

Decoding

6

Decoding

• Encoding by itself is useless

6

Decoding


• Decoding is also necessary

6

Decoding



• So… we need more bits

6

Decoding




• How many bits do we need?

6

Decoding





• lowercase alphabet

6

Decoding





• lowercase alphabet

• 5 bits

6

A Second Attempt

7

A Second Attempt

• Let’s try another encoding

7

A Second Attempt

• Let’s try another encoding

7


m 01100 z 11001

Analysis

8

Analysis

Pros

8

Analysis

Pros


8

Analysis

Pros


• Decoding was possible

8

Analysis

Pros



Cons

8

Analysis

Pros



Cons

• Takes more space…

8

Analysis

Pros



Cons


• What restriction did we place that’s unnecessary?

8

Analysis

Pros



Cons


• What restriction did we place that’s unnecessary?

• Fixed length

8

Variable Length Encoding

9


• Problems?

9


• Problems?

• When do we start and stop?

9


• Problems?


• String of As and Bs: ABA

9


• Problems?



• A - 00, B - 0

9


• Problems?



• A - 00, B - 0

• Encode ABA: 00000

9


• Problems?



• A - 00, B - 0


• Decode 00000:

9


• Problems?



• A - 00, B - 0


• Decode 00000:

• ABA, AAB, BAA?

9


• Problems?



• A - 00, B - 0


• Decode 00000:

• ABA, AAB, BAA?

• What lengths do we use?

9

A Second Look at Fixed Length

10

A Second Look at Fixed Length

10


m 01100 z 11001

11

Trees!

11

Trees!

Letter Binary

A 00

B 01

C 1

11

Trees!

0 1

Letter Binary

A 00

B 01

C 1

11

Trees!

0 1

C

Letter Binary

A 00

B 01

C 1

11

Trees!

0 1

0 1C

Letter Binary

A 00

B 01

C 1

11

Trees!

0 1

B

0 1C

Letter Binary

A 00

B 01

C 1

11

Trees!

0 1

A B

0 1C

Letter Binary

A 00

B 01

C 1

12

What happens when…?


m 1 z 0

12



m 1 z 0

• Rule 1: Each leaf only has 1 label


13

Letter Binary

A 00

B 0


13

• Rule 2: Only leaves get labels

Letter Binary

A 00

B 0

An Optimal Encoding

14

An Optimal Encoding

14

• Start with a tree

0 1

A B

0 1C

An Optimal Encoding

14


• What kinds of things do we want to encode with this?

0 1

A B

0 1C

An Optimal Encoding

14



• What letter do we want to appear the most?

0 1

A B

0 1C

An Optimal Encoding

14




• How about the least?

0 1

A B

0 1C

An Optimal Encoding

14




• How about the least?

• This is called a Huffman Encoding

0 1

A B

0 1C

Huffman Encoding

15

Huffman Encoding

15

• Let’s pretend we want to come up with the optimal encoding:

Huffman Encoding

15


• AAAAAAAAAABBBBBCCCCCCCDDDDDDDDD

Huffman Encoding

15



• A appears 10 times

Huffman Encoding

15




• B appears 5 times

Huffman Encoding

15





• C appears 7 times

Huffman Encoding

15






• D appears 9 times

Huffman Encoding

16

Huffman Encoding

16

• Start with the two smallest frequencies

Huffman Encoding

16


• A appears 10 times, B appears 5 times, C appears 7 times, D appears 9 times

Huffman Encoding

16



A

B

C

D

Huffman Encoding

16



A

B

C

D

Huffman Encoding

16



A

B

C

D

0 1

A

D

B C

Huffman Encoding

17

Huffman Encoding

17

• Continue…

Huffman Encoding

17

• Continue…

• A appears 10 times, B & C appear a combined 12 times, D appears 9 times

Huffman Encoding

17

• Continue…


0 1

A

D

B C

Huffman Encoding

17

• Continue…


0 1

A

D

B C

Huffman Encoding

17

• Continue…


0 1

A

D

B C

0 1

B C

0 1

A D

Huffman Encoding

18

Huffman Encoding

18

• And finally…

Huffman Encoding

18

• And finally…

0 1

B C

0 1

A D

Huffman Encoding

18

• And finally…

0 1

B C

0 1

A D

Huffman Encoding

18

• And finally…

0 1

B C

0 1

A D

B D

0 1

C

0 1

A

0 1

Huffman Encoding

19

Huffman Encoding

19

• Another example…

Huffman Encoding

19


• AAAAAAAAAABCCD

Huffman Encoding

19


• AAAAAAAAAABCCD


Huffman Encoding

19


• AAAAAAAAAABCCD


• B appears 1 time

Huffman Encoding

19


• AAAAAAAAAABCCD




Huffman Encoding

19


• AAAAAAAAAABCCD




• D appears 1 time

Huffman Encoding

20

Huffman Encoding

20


Huffman Encoding

20


• A appears 10 times, B appears 1 time, C appears 2 times, D appears 1 time

Huffman Encoding

20



A

B

C

D

Huffman Encoding

20



A

B

C

D

Huffman Encoding

20



A

B

C

D

0 1

A

C

B D

Huffman Encoding

21

Huffman Encoding

21


Huffman Encoding

21


• A appears 10 times, B & D appear a combined 2 times, C appears 2 times

Huffman Encoding

21



0 1

A

C

B D

Huffman Encoding

21



0 1

A

C

B D

Huffman Encoding

21



0 1

A

C

B D

0 1

C 0 1

B D

A

Huffman Encoding

22

Huffman Encoding

22

• And finally…

Huffman Encoding

22

• And finally…

0 1

C 0 1

B D

A

Huffman Encoding

22

• And finally…

0 1

C 0 1

B D

A

Huffman Encoding

22

• And finally…

0 1

C 0 1

B D

0 1

A

0 1

C 0 1

B D

A

Documents

61A Extra Lecture 4