T.Sharon-A.Frank

Multimedia

Discrete Media: Text, Still Images

T.Sharon-A.Frank

2

Contents

• Discrete media types:– Text

– Still images:• Vector Graphics

• Bitmapped Images

• Continuous media types:– Audio

– Moving images:• Animations

• Video

• Media/Tools Comparisons

T.Sharon-A.Frank

3

Capture vs. Synthesize

memo

Capture

A/DRecognition

Computer synthesis

Animation

Video

VectorGraphics

Text

Images

Audio

Analogvideocamera

Electronic pen

Printeddocuments

Sound

T.Sharon-A.Frank

4

Conventional Classification of Media Types

Audio Video Animation

TextVector

GraphicsImages

Capturedfrom real world

Synthesizedby computers

Continuous(time based)

Discrete(space based)

Remark: Conventionally for Origin means usually, not always

Origin

Time/space nature

T.Sharon-A.Frank

5

Synthesis vs. Captured

T.Sharon-A.Frank

6

Text

• Formats– Plain (ASCII)

– Rich text (Word, Latex, HTML)

• Issues– Tags

– Special characters

– Bilingual

• Creation– Mostly synthesized (keyboard, OCR)

• Presentation

T.Sharon-A.Frank

7

Text in Multimedia

• Text is critical to communication even in multimedia.

• Hypertext uses text but is fundamentally different than traditional text.

• Choice of words is very important and should be done carefully.

T.Sharon-A.Frank

8

Text File Size

• 2 bytes per character

• Each character 8x8 pixels (picture elements)

• Characters per page = = 4,800

• Storage required = 4,800 * 2 = 9,600 bytes

Total: ~9.4KB

640 * 480

8* 8

T.Sharon-A.Frank

9

• Production and display of still images stored in digital form:– Digitize printed image with a scanner

– Capture image from digital camera

– Grab frame from video camera

– Create in digital form using graphics package

– Generate visual representation of data.

Computer Graphics

T.Sharon-A.Frank

10

Use of Image/Picture

1. Provide facts2. Explain a process3. Set a mood4. Evoke an idea5. Pinpoint location6. Illustrate relationships7. Tell stories8. Identify/Compare entities

T.Sharon-A.Frank

11

• Many graphics file formats in existence.

• Different ways of encoding image data.

• Different amounts/form of supplementary data.

• Different compression methods for images:– Lossless: image can be reconstructed exactly from

compressed version.– Lossy: some information discarded; image can

only be reconstructed approximately.

Graphics File Formats

T.Sharon-A.Frank

12

• Image is displayed on monitor as array of pixels (picture elements)– Rectangular (usually square) dots of color.

• Program (e.g., Web browser) sets pixels to an appropriate color to produce desired image– Pixels merge optically to produce effect of

continuous tone.

• Program must maintain a model of the image– May be stored in a file and read by program.

Rendering

T.Sharon-A.Frank

13

• Vector graphics – Image is modelled as mathematical description of curves and shapes:– e.g., a vector is a line that is described by the

location of its two endpoints.– Rendered by computing pixels from description.

• Bitmapped graphics – Image is modelled as a matrix/grid of pixel values– Rendered by direct mapping of logical pixels to

physical pixels of screen, may need scaling and clipping.

Vector and Bitmapped Graphics

T.Sharon-A.Frank

14

Graphics (Vector) vs. Images (Bitmapped)

• Revisable documents• Format can retain

structural information• Semantic content is

preserved in the representation

• Described as objects

• Not revisable• Format unaware of

structural information• Semantic content is

not preserved• Described as bitmaps

formed of individual pixels

A

T.Sharon-A.Frank

15

Vector Graphics

• Format of Vector Graphics is:– Compact – Scalable– Resolution-independent

• Hence it is attractive for networked multimedia.

• Widely used on Web, e.g., SWF and SVG.

T.Sharon-A.Frank

16

• EPS (Encapsulated PostScript)– Primarily print, use declining, superseded by PDF.

• SVG (Scalable Vector Graphics)– W3C standard, still needs to be widely used.

• SWF (Shockwave Flash)– From Adobe Flash; de facto standard; primarily for

vector animation, but can be used for vector graphics.

• XAML (Extensible Application Markup Language) – From Microsoft Silverlight; used for marking up

vector graphics and animations.

Sample Vector Graphics Formats

T.Sharon-A.Frank

17

Bitmapped Image

• An image is a spatial representation of an object.

• An image can be represented as a matrix/grid of pixels.

• Records a value for every pixel in the image.

• Also known as raster graphics.

T.Sharon-A.Frank

18

Bitmapped Image Representation

• An image can be thought of as a function with resulting values of the light intensity at each point over a planar region.

• In computers, this function must be sampled at discrete intervals.

• Intensity at each pixel is represented by an integer into which the color of the image is encoded.

• If there are just 2 intensity values, 0 and 1 represent black and white (1 bit).

• If 8-bit integers, color levels range from 0 (black) to 255 (white).

T.Sharon-A.Frank

19

Image Characteristics

• Image/Device Resolution (ppi/dpi)

• Color Depth (1 bit and halftone, grayscale, color, true color) – bits per pixel

• Color Model (RGB, CMY, YUV)

T.Sharon-A.Frank

20

• Printers, scanners: specified as dots per unit length, often dots per inch (dpi):– Desktop printer 600dpi, typesetter 1270dpi,

scanner 300–3600dpi, …

• Video, monitors: specified as pixel dimensions:– PAL TV 768x576px, 17" CRT monitor

1024x768px, …– dpi depends on physical size of screen.

Device Resolution

T.Sharon-A.Frank

21

• Bitmaps are resolution dependent.

• Array of pixels has pixel dimensions, but no physical dimensions.

• By default, displayed size depends on resolution (dpi) of output device– physical dimension = pixel dimension/resolution.

• Can store image resolution (ppi) in image file to maintain image's original size– Scale by device resolution/image resolution.

Image Resolution

T.Sharon-A.Frank

22

Commonly used Color Models

• RGB: Red, Green, Blue – additive primary colors (color monitors).

• CMY: Cyan, Magenta, Yellow – subtractive primary colors plus black (color printers).

• YIQ (YUV): luminance, chrominance are decoupled (TV).

• HSV: Hue, Saturation and Value (color image manipulation).

T.Sharon-A.Frank

23

Image Format

• A stored image is an array of values:– Each value represents data associated with a pixel

in the image.– For color, this value may be:

• 3 numbers – intensities of red, green and blue components of color at that pixel

• 3 numbers – indices to tables of RGB intensities

• 1 number – index to table of color triples

• 1 number – index to other color models, such as CMY, YIQ, HSV, etc.

T.Sharon-A.Frank

24

Sample Image File Formats

• BMP (Bit Map Picture)

• TIFF (Tagged Image File Format)

• GIF (Graphics Interchange Format)– Lossless, 256 colors (indexed), transparency

• PNG (Portable Network Graphics)– Lossless, variable number of colors, W3C standard

• JPEG (Joint Photographic Experts Group)– Lossy (variable quality), millions of colors

T.Sharon-A.Frank

25

GIF (Graphic Interchange Format)

• Most popular image format on Web.

• Good for condensing rows of pixels of identical color: logos, icons, line art, cartoons, etc.

• Compression is lossless since uses LZW algorithm which takes advantage of repetition in data streams.

• 8-bit indexed color – indexed means each pixel contains index to cell in color table.

T.Sharon-A.Frank

26

More about GIF

• Interlacing:

– normal GIFs are displayed either 1 row of pixels at a time or all at once when entire image is downloaded.

– Interlaced GIFs displayed in 4 passes – every 4th line in each pass.

• GIF87a vs. GIF89a

– GIF89a allows animation and transparency.

T.Sharon-A.Frank

27

JPEG (Joint Photographic Experts Group)

• Compression is lossy:– based on spatial frequency.– low frequency areas compressed more

efficiently than high frequency areas.• Good for photographic images (better than GIF

compression anyway):– how much compression depends on photo.– e.g., detail-rich photo takes 85K as GIF but

requires only 35K as JPEG.

T.Sharon-A.Frank

28

More about JPEG

• Allows variable compression:– Q setting between 1 and 100 (usually)

varies among tools that create JPEGs– lower number = lower image quality but

better compression rates (smaller files)– Q setting has no mathematical significance– actual compression ratio depends on

individual image.• Lossy compression makes flat colors blotchy

and pixilated.

T.Sharon-A.Frank

29

PNG (Portable Network Format)

• Developed in early 1995 as non-proprietary alternative to GIF.

• Taking a while to catch on.• Uses lossless compression:

– horizontal compression similar to GIF– vertical compression using series of filters– typically compresses 5-25% better than GIF.

T.Sharon-A.Frank

30

More about PNG

• A good GIF substitute.• Not a good JPEG substitute (lossless

compression results in bigger files than JPEGs lossy compression).

• Supports 24 bit indexed color (and higher so could be a good TIFF substitute for detailed photos).

• Supports interlacing, transparency, and lots of other unique features.

T.Sharon-A.Frank

31

Image Acquisition

• Capturing (as bitmaps)– scanner– digital camera– Videotape/Analog camera + video card

• Synthesizing– Drawing programs (Adobe Illustrator, Macromedia

Freehand, Paint Shop Pro, MS Office, etc)– Image-editing software (MS Paint, Adobe

Photoshop, Photoshop Elements, ImageReady, GIMP, etc)

T.Sharon-A.Frank

32

Image Presentation

• Computer Display

• PDA screens

• Printers

T.Sharon-A.Frank

33

• Bitmapped – any picture of w x h pixels, using c bytes per pixel, occupies w x h x c bytes.

• Vector – space required depends on complexity of picture (how many shapes, segments of path, etc).• Usually vector graphics memory requirements are smaller than those of bitmapped – when few vectors, more efficient than bitmaps.

Memory Requirements

T.Sharon-A.Frank

34

Vector Graphics Size

• Typical screen – 500 elements.• Say each element has:• 1 attribute/color field – 8 bit

• 2 points containing:

• Bits per element: 2*(9+10)+8 = 46 bits• Storage required per screen =

500*46/8 = 2,875 bytesTotal: ~2.8KB

• x position – 10 bit (log2640)• y position – 9 bit (log2480)

T.Sharon-A.Frank

35

Color Raster Picture Size

• One screen: 640 * 480 = 307,200 pixels

• 256 colors – 1 byte per pixel = 307,200 bytes

Total: ~300KB

• True color (16M colors) – 3 bytes per pixel =

307,200*3 = 921,600 bytes

Total: ~900KB