A Cultural History of JPEG-1 and a Recent...

January 30, 2008

A Cultural History of JPEG-1 and a Recent EnhancementDr. Joan L. Mitchell, FellowJanuary 30, 2008

2

ABSTRACT:

The "J" in JPEG (Joint Photographic Experts Group) acknowledges its two main parent organizations, ISO (International Organization for Standardization) and CCITT (International Telegraph and Telephone Consultative Committee) which is now called ITU (International Telecommunications Union). The JPEG committee was one of the first times these organizations formed a single committee for the purpose of creating a joint standard despite their different histories, cultures and procedures for settling disputes. Rather than trying to meld its parents' cultures or deciding which set of rules to use for settling conflicts, the original JPEG committee adopted a 100% consensus rule. The first part of this talk will share stories illustrating how achieving complete consensus influenced the definition of this widely-used standard.

The second part of this talk will review the technologies in JPEG-1 (DCT, ADPCM, Huffman, and arithmetic coding). Then it will explain a recent enhancement, ITU-T Rec. T.851. This additional arithmetic coder has minimum latency which makes it particularly suitable for the mobile phone industry.

3

OUTLINE:

o Standards organizations involved

o International effort

o Stories illustrating JPEG culture

o Review technologies

o Recent enhancement

4

Standards Organizations involved:

o Int’l Telegraph and Telephone Consultative Committee (CCITT) - International Telecommunications Union (ITU)

* ITU-T (Terminals) formerly CCITT* ITU-R (Radio) formerly CCIR

o Int’l Organization for Standardization (ISO)

o Int’l Electrotechnical Commission (IEC)

5

ITU-T (formerly CCITT):

o UN sponsored treaty organizationo Official voting done by nationso Issues "Recommendations"o Voluntary for nations to adopt

- Often made law of the lando If controversial, study it moreo Study Groups (SG)

- Subcommittees (SC) study questions

6

ISO

o Non-treaty agency of the United Nationso Works for good of the industryo Decentralized & self-regulatingo “Let the market decide”o Technical committees (TC)

- Study Committees (SC)* Working Groups (WG)

Experts Groups (EG)

7

JPEG (Joint Photographic Experts Group)

o CCITT SG8- Special Rapporteur Group on New Forms

of Image Communicationo ISO

- ISO/TC97/SC2/WG8- ISO/IEC JTC1/SC2/WG8- ISO/IEC JTC1/SC2/WG10- ISO/IEC JTC1/SC29/WG10- ISO/IEC JTC1/SC29/WG1 (with JBIG)

8

International Effort (11 countries):

o North America- USA, Canada

o Europe- Denmark, France, Germany, TheNetherlands, United Kingdom

o Middle East- Israel

o Far East- Japan, Republic of Korea

9

Meeting Locations (’87-’88):

o Darmstadt, Germany Mar. ’87o Copenhagen, Denmark June ’87o Washington, DC USA Oct. ’87o London, UK Dec. ’87o Copenhagen, Denmark Jan. ’88o Ottawa, Canada May ’88- Ad Hoc Rennes, France July ’88- Ad Hoc Torino, Italy Sept. ’88o London, UK Sept. ‘88

10

Meeting Locations (’89-’90):

o Livingston, NJ USA Feb. ’89o Stockholm, Sweden July ’89o Tokyo, Japan Oct. ’89o Tampa, FL USA Mar. ’90- Ad Hoc Budapest, Hungary Apr. ’90o Porto, Portugal July ’90- Ad Hoc editing Cambridge, MA Oct. ’90o Later meetings in Israel and Korea

11

JPEG Culture:

o Adopted 100% consensus ruleo How reach consensus?

- Run a contest w/ objective criteria- Create an ad hoc- Find a “third answer”- Head of Delegations (HOD) meet- National delegation caucus - Vote late in day

12

NEVER LEAVE A MEETING EARLY (if you care about the results)

o On plane discovered registration deadline

o Wrote IBM proposal as others presented

13

PUT IT IN WRITING:

o “If it ain’t in writing, it ain’t.”- Listen politely- Completely ignore

o English was official language- 4 native English speakers (after a year)

o Often technical experts not at meeting- Written information faxed home- May not understand fast American

14

“If I gore your ox today, someday you will gore my ox.”

o One proposal was best & worsto Had used test set to derive tableso Never censored/condemned/embarrassedo Also never mentioned again

15

“It’s Time for the Third Answer”

o Standards’ motto: “Equal pain”o I proposed: “Equal gain”o Two groups butting headso Their views form triangle baseo Now lift both up and find common groundo Third answer better for both

16

One day the smoke disappeared.

o Traditionally several ash trays per tableo More than 1/3 (some chain) smokedo Bill realized “Nobody is smoking.”o Since last meeting Japan had declared

all business meetings smoke free.o No ash trays on table

17

IT’S JUST A GAME! Or is it?

Our motto: Good in hardware, Good in software, And the best.We/Everyone wins

18

Have faith, we have never left unhappy.

o Almost every meeting was a cliff hanger

o Ex: 5pm Friday results appeared poor.- Objective clear winner worst quality.- 7pm threw out objective criteria

19

Blur Lines:

o Picked 3 techniques from June’87 judging

o Formed ad hocs teams to “refine” method- Multiple companies- Multiple continents- Multiple nations

20

Once you agree, unless some new information, the answer won’t change.

o Editing and/ ad hoc reaches consensus.

o Written version distributed.

o JPEG committee gives approval

o Only new information can change answer.

21

HOW CAN WE TELL THEM?

o Header more complex than algorithm

o Half already knew

o Major revision of header structure

22

Ad Hoc Group on Front End & Back End Processing:

o Agreed to one technologyo Took home tape of resultso Horrid on computer monitorso Committee assumed sour grapeso Week long visits to NY/Denmark to solveo AC prediction removes “blockiness”

23

ALWAYS THANK THE MEETING HOSTS

o Formal resolution put in minutes

o Invite meeting coordinators into room

o Read them the resolution

o Everyone claps loudly in appreciation

o Meeting adjourns

24

Recent enhancement:

Intended for mobile phone standard

Complexity Power Battery life

Compression Transmission time – Storage costs

25

TECHNOLOGY REVIEW:

Discrete Cosine Transform (DCT)

Quantization (source of most of loss)

Differential Pulse Code Modulation (DPCM)

Entropy coding– Huffman coding– Arithmetic coding

26

What is in JPEG Part 1?

Baseline sequential (lossy)– 8 bits/sample– 8x8 DCT– Huffman only – Required in all DCT-based decoders Extended sequential (lossy) option

– 8 or 12 bits/sample– 8x8 DCT– Huffman or arithmetic coding (QM-coder)

27

JPEG Part 1:

Progressive (lossy)– 8 or 12 bits/sample– 8x8 DCT– Huffman or arithmetic coding (QM-coder)– SA, SS, and/or hierarchical Lossless

– 2 to 16 bits/sample– Predictive based (DPCM) – Huffman or arithmetic coding (QM-coder)– Sequential and/or hierarchical

28

DCT Transform coding (Zig-zag order)

* DC term. Next 63 are AC terms

29

JPEG example quantization tables (dark large)

30

JPEG Progressive vs. Sequential:

Sequential: 1 pass to final qualityProgressive : Multiple passes to final quality– DC always separate pass(es)– Spectral selection (SS): band of coefficients– Successive approximation (SA)

• 1st pass missing some lsb of AC terms• 1 bit added per later pass(es)

– Can mix SS and SAFollowing images created by W.B. Pennebaker

31

DC without lsb (0.088b/p)

Entropy coder is arithmetic coding. Sequential is to nearest 8 line boundary.

32

+1-5 AC without 3 lsb (0.202 b/p)

33

+6-10 AC without 3 lsb (0.261 b/p)

34

+11-63 AC without 3 lsb (0.283 b/p)

35

+1-63 AC 3rd lsb (0.482 b/p)

36

+1-63 AC 2nd lsb (0.796 b/p)

37

DC +1-63 AC lsb (1.379 b/p)

Original = 512 x 512 bytes

38

JPEG Predictors:

Px = none (difference image)Px = RaPx = RbPx = RcPx = Ra – Rb + RcPx = Ra + (Rb – Rc)/2Px = Rb + (Ra – Rc)/2 Px = (Ra + Rb)/2

c ba x

39

Entropy Coding:

Traditionally losslessHuffman coding– Variable length codes– Integer bits– More complex modelsArithmetic coding– IBM invention– Dynamic probabilities– Better compression

40

Creation of a Huffman Code:

Given four symbols with probabilities 1/2, 1/4, 1/8, and 1/8, create a tree by aggregating the two smallest probabilities at each stage.

Then assign a "0" or "1" to each decision.

111

110

10

0

1

0

1/8

1/2

1/4

1/2

0

0

1

1

1/4

1/8

A

B

C

D

41

Variable Length Decoding:

Decoding by table lookup: bits s L(s) 000 A 1 001 A 1 010 A 1 011 A 1 100 B 2 101 B 2 110 C 3 111 D 3

start

bit?

bit?

bit?

D C

B

A

Finite-state-machine decoding:

42

Arithmetic Coding:

Binary fraction

Elias coding (student exercise)

Carry over problem

43

Binary Fraction between 0 & 1

Take number line from [0 to 1)Binary point before bit streamsDescribe an interval on number lineEnough bits to get uniquely inside

– No outside fraction has same prefix– Two extra bits guarantee inside

44

Elias Coding (aka Jerry Goertzel):

Nested sub-intervalsFinal interval size = Product of probabilities Log (Product P(i))= Sum Log P(i)Student exercise

– Draw number line from 0 to 1– Pick probabilities for symbols A, B, C– Partition line proportional to probabilities– Pick one and partition again

45

Carry over problem:

Ex: Point below at 0.0111111111.... with top of range above 0.10000000...

Resolving carry in encoder requires latency

Can resolve carry in decoder– Stuff a bit if byte 0xFF

46

Recent Enhancement:

o Optional alternative arithmetic coder

o Q15-coder

o Fee-free

o Baseline decoder not mandatory

o Converts into baseline with no loss

47

QM- coder MQ-coder

(JPEG-1 & JBIG-1)SW conventionsEnc. resolves carry– Latency potentially rest of

data stream– Byte stuffing 0xFF-> 0xFF00

Starts with 0xFFD8.jpg file extensionFixed fee

(JPEG2000 & JBIG-2)HW conventionsDec. resolves carry

Latency 1 byteBit stuffing

Fee-free.jp2 file extension

48

MQ- coder Q15- coder

(JPEG2000 & JBIG-2)HW conventionsDec. resolves carry– Latency 1 byte– Bit stuffing

Fee-free.jp2 file extension

(ITU-T Rec. T.851)SW conventionsDec. resolves carry

– Latency 1 byte– Bit stuffing

Starts with 0xFFC8 0005 ‘ac2’

Fee-free.jpe file extension (?)

49

Compression Performance:

10-20 % improvement over baseline

+/- 0.5% compared to QM-coder

Beta IJG software on ITU-T website

January 30, 2008

Thank You