7MULTIMEDIA OPERATING SYSTEMS

7.1 INTRODUCTION TO MULTIMEDIA

7.2 MULTIMEDIA FILES

7.3 VIDEO COMPRESSION

7.4 MULTIMEDIA PROCESS SCHEDULING

7.5 MULTIMEDIA FILE SYSTEM PARADIGMS

7.6 FILE PLACEMENT

7.7 CACHING

7.8 DISK SCHEDULING FOR MULTIMEDIA

7.9 RESEARCH ON MULTIMEDIA

7.10 SUMMARY

Distribution network

Distribution network

Fiber

Video server

Video server

Copper twisted pair

Junction box

Junction box

House

Cable TV coaxial cable

Fiber

(a)

(b)

Fig. 7-1. Video on demand using different local distribution tech-nologies. (a) ADSL. (b) Cable TV.

22222222222222222222222222222222222222222222222222Source Mbps GB/hr22222222222222222222222222222222222222222222222222

Telephone (PCM) 0.064 0.0322222222222222222222222222222222222222222222222222MP3 music 0.14 0.0622222222222222222222222222222222222222222222222222Audio CD 1.4 0.6222222222222222222222222222222222222222222222222222MPEG-2 movie (640 × 480) 4 1.7622222222222222222222222222222222222222222222222222Digital camcorder (720 × 480) 25 1122222222222222222222222222222222222222222222222222Uncompressed TV (640 × 480) 221 9722222222222222222222222222222222222222222222222222Uncompressed HDTV (1280 × 720)648 2882222222222222222222222222222222222222222222222222211111111111

11111111111

11111111111

11111111111

222222222222222222222222222222Device Mbps222222222222222222222222222222

Fast Ethernet 100222222222222222222222222222222EIDE disk 133222222222222222222222222222222ATM OC-3 network 156222222222222222222222222222222SCSI UltraWide disk 320222222222222222222222222222222IEEE 1394 (FireWire) 400222222222222222222222222222222Gigabit Ethernet 1000222222222222222222222222222222SCSI Ultra-160 disk 128022222222222222222222222222222211111111111

11111111111

11111111111

Fig. 7-2. Some data rates for multimedia and high-performanceI/O devices. Note that 1 Mbps is 106 bits/sec but 1 GB is 230

bytes.

1 432 5 6 7 8

Hello, Bob Hello, Alice Nice day Sure is How are you Great And you Good

Dag, Bob Dag, Alice Mooie dag Jazeker Hoe gaat het Prima En jij Goed

Video

English audio

French audio

German audio

English subtitles

Dutch subtitles

Fast forward

Fast backward

Frame

Fig. 7-3. A movie may consist of several files.

1.00

0.75

0.50

0.25

0

–0.25

–0.50

–0.75

–1.00

12 T

12 T

T T T

(a) (b) (c)

12 T

Fig. 7-4. (a) A sine wave. (b) Sampling the sine wave. (c) Quantiz-ing the samples to 4 bits.

Scan line

1

3

5

7

9

11

13

15

483

Tim

e

.

.

.

The next fieldstarts here

Scan line paintedon the screen

Horizontalretrace

Verticalretrace

Fig. 7-5. The scanning pattern used for NTSC video and televi-sion.

480

640

(a) (b) Q

RGB Y I640

480

240

320

240

1 Block

Block 4799

8-Bit pixel

24-Bit pixel

Fig. 7-6. (a) RGB input data. (b) After block preparation.

Y/I/

Q A

mpl

itude

DC

T

x Fx

y Fy

Fig. 7-7. (a) One block of the Y matrix. (b) The DCT coefficients.

150

92

52

12

4

2

1

0

80

75

38

8

3

2

1

0

40

36

26

6

2

1

0

0

14

10

8

4

0

1

0

0

4

6

7

2

0

0

0

0

2

1

4

1

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

DCT Coefficients

150

92

26

3

1

0

0

0

80

75

19

2

0

0

0

0

20

18

13

2

0

0

0

0

4

3

2

1

0

0

0

0

1

1

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Quantized coefficients

1

1

2

4

8

16

32

64

1

1

2

4

8

16

32

64

2

2

2

4

8

16

32

64

4

4

4

4

8

16

32

64

8

8

8

8

8

16

32

64

16

16

16

16

16

16

32

64

32

32

32

32

32

32

32

64

64

64

64

64

64

64

64

64

Quantization table

Fig. 7-8. Computation of the quantized DCT coefficients.

150

92

26

3

1

0

0

0

80

75

19

2

0

0

0

0

20

18

13

2

0

0

0

0

4

3

2

1

0

0

0

0

1

1

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Fig. 7-9. The order in which the quantized values are transmitted.

Fig. 7-10. Three consecutive video frames.

A1 A2 A3 A4 A5

B1 B2 B3 B4

Starting momentfor A1, B1, C1

Deadlinefor A1 Deadline for B1

Deadline for C1

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140

Time (msec)

A

B

C C2 C3C1

Fig. 7-11. Three periodic processes, each displaying a movie. Theframe rates and processing requirements per frame are differentfor each movie.

A1

A1

A1

B1

B1

A2

A2

A2

A3

A3 B3 A4

A3 B3 A4 A5 B4

A5 B4

A4 A5

B1 B2

B2

B2

B3 B4

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140

Time (msec)

A

B

C

EDF

RMS

C1

C1

C1

C2

C2

C2

C3

C3

C3

Fig. 7-12. An example of RMS and EDF real-time scheduling.

A1

A1

B1

B1

A1

A2

B2 B3A3 A4 A5 B4

A5

B1 B2

B2 Failed

A2

B3 B4

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140

Time (msec)

A

B

C

EDF

RMS

A2 A3 A4

C2 C3

C3

C1

C1 C2

Fig. 7-13. Another example of real-time scheduling with RMS andEDF.

Videoserver Client

Request 1

Request 2

Block 1

Block 2

Request 3

Block 3

Tim

e

(a)

Videoserver Client

Start

Block 1

Block 2

Block 3

Block 4

Block 5

(b)

Fig. 7-14. (a) A pull server. (b) A push server.

00 9000 18000 27000 36000 45000 54000 63000 72000 81000

8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45

1 0 9000 18000 27000 36000 45000 54000 63000 72000

02 9000 18000 27000 36000 45000 54000 63000

03 9000 18000 27000 36000 45000 54000

04 9000 18000 27000 36000 45000

05 9000 18000 27000 36000

06 9000 18000 27000

07 9000 18000

08 9000

09

Frame 9000 instream 3 is sent

at 8:20 min

Time

Stream

Fig. 7-15. Near video on demand has a new stream starting at reg-ular intervals, in this example every 5 minutes (9000 frames).

300 60 90 120

Play point at 12 min

Play point at 75 min

Play point at 15 min

Play point at 16 min

Play point at 22 min

(a)

(b)

(c)

(d)

(e)

Minutes

Fig. 7-16. (a) Initial situation. (b) After a rewind to 12 min.(c) After waiting 3 min. (d) After starting to refill the buffer.(e) Buffer full.

Video A A A T T Video A A A T T Video A A A T T

Frame 3Frame 2Frame 1

Audiotrack

Texttrack

Fig. 7-17. Interleaving video, audio, and text in a single contigu-ous file per movie.

Disk block largerthan frame

AudioText

I

I

FrameIndex

Disk block smallerthan frame

I

BlockIndex

I

I

I

I

(a) (b)

I-frame P-frameUnused

Fig. 7-18. Noncontiguous movie storage. (a) Small disk blocks.(b) Large disk blocks.

0 9000 18000 27000 36000 45000 54000 72000 81000 20700063000

Stream24

Stream23

Stream15

Stream1

1 9001 18001 27001 36001 45001 54001 72001 81001 20700163001

2 9002 18002 27002 36002 45002 54002 72002 81002 20700263002

Frame 27002 (about 15 min into the movie)

Track 1

Track 2

Track 3

Order in which blocks are read from disk

Fig. 7-19. Optimal frame placement for near video on demand.

0.300

0.250

0.200

0.150

0.100

0.050

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Rank

Freq

uenc

y

Fig. 7-20. The curve gives Zipf’s law for N = 20. The squaresrepresent the populations of the 20 largest cities in the U.S., sortedon rank order (New York is 1, Los Angeles is 2, Chicago is 3,etc.).

Movie10

Movie8

Movie6

Movie4

Movie2

Movie1

Movie3

Movie5

Movie7

Movie9

Movie11

Cylinder

Freq

uenc

y of

use

Fig. 7-21. The organ-pipe distribution of files on a video server

A0A1A2A3A4A5A6A7

B0B1B2B3B4B5B6B7

C0C1C2C3C4C5C6C7

D0D1D2D3D4D5D6D7

(a)

A0A4B0B4C0C4D0D4

A1A5B1B5C1C5D1D5

A2A6B2B6C2C6D2D6

A3A7B3B7C3C7D3D7

(b)

A0A4B3B7C2C6D1D5

A1A5B0B4C3C7D2D6

A2A6B1B5C0C4D3D7

A3A7B2B6C1C5D0D4

(c)

A0A6B3B4C0C7D1D6

A2A5B1B7C2C6D2D5

A1A4B2B5C3C4D3D4

A3A7B0B6C1C5D0D7

(d)

1 2 3 4 1 2 3 4

1 2 3 4 1 2 3 4

Disk

Fig. 7-22. Four ways of organizing multimedia files over multipledisks. (a) No striping. (b) Same striping pattern for all files.(c) Staggered striping. (d) Random striping.

10 sec 1 min 2 min 3 min 4 min

0

1800

3600

5400

1800

3600

5400

7200

7200

User 1

0

User 2

Starts10 seclater

Time

0

1800

3600

5400

7200

User 1

User 2

Runs faster Normal speed

(a)

(b)

Runs slower Normal speed

5400

7200

1800

36000

Fig. 7-23. (a) Two users watching the same movie 10 sec out ofsync. (b) Merging the two streams into one.

1

701

2

92

3

281

4

130

5

326

6

410

7

160

8

466

9

204

10

524

92 130 160 204 281 326 410 466 524 701

Stream

Optimization algorithm

Order in which disk requests are processed

Buffer for odd framesBuffer for even frames

Block requested

Fig. 7-24. In one round, each movie asks for one frame.

Requests (sorted on deadline)

676

700 710 720 730 740 750

330

110 680 440 220 755 280 550 812 103

Deadline (msec)

Cylinder

Batch together

Fig. 7-25. The scan-EDF algorithm uses deadlines and cylindernumbers for scheduling.