Upload
saul
View
52
Download
0
Embed Size (px)
DESCRIPTION
Video As A Datatype. Ketan Mayer-Patel. The mothers of all video. NTSC - American standard PAL - European standard SECAM - French standard (ugly stepchild) For better or worse, this is where it all starts. Color Spaces. One luminance component Two chrominance components - PowerPoint PPT Presentation
Citation preview
Comp 294-9 :: Fall 2003
Video As A Datatype
Ketan Mayer-Patel
Comp 294-9 :: Fall 2003
The mothers of all video...• NTSC - American standard• PAL - European standard• SECAM - French standard (ugly stepchild)• For better or worse, this is where it all starts.
Comp 294-9 :: Fall 2003
Color Spaces• One luminance component• Two chrominance components
– Original TV was black and white.– Adding color had to be done in a compatible way.
• NTSC: YIQ• PAL: YUV• In general: YUV and YCrCb used as terms.
Comp 294-9 :: Fall 2003
YUV vs. RGB• Simple linear transform from one cube to
the other.• Specific transform coefficients vary for
specific systems, but a common one:– Y = 0.299 R + 0.587 G + 0.114 B– Cr = -0.147 R - 0.289 G + 0.437 B + 0.5– Cb = 0.615 R - 0.515 G - 0.100 B + 0.5
Comp 294-9 :: Fall 2003
YUV Challenges• YUV is like taking RGB cube, standing it
on a corner with (0,0,0) on bottom and (1,1,1) at top, rotating slightly, and taking the bounding box.
• What problems might this incur?– Some RGB colors are illegal.– A lot of YUV colors are illegal.
Comp 294-9 :: Fall 2003
A Note On Color• Previous view of color is EXTREMELY
simplified.– Color is complicated
• Frequency dependent response for contrast, lightness, etc.
– Gamma correction.`
Comp 294-9 :: Fall 2003
Scanning• Analog video signal is continuous voltage
signal that gets scanned along the screen.• The electron stream controlled by two
orthogonal sets of magnets.– Horizontal: Beam is moved from left to right
and then quickly back.– Vertically: Beam is moved from top to bottom
and then quickly back.
Comp 294-9 :: Fall 2003
Scanning illustrated
Time
Vol
tage
Comp 294-9 :: Fall 2003
Structure of Video• Blinn’s article.
– Macrostructure• Frequency peaks at frame rate.
– Microstructure• Frequency peaks at line rate
• Adding color is a good first example of source-aware channel coding.– What was the problem?– What was the solution?– Why does it work?– What are its drawbacks (i.e., when does it fail)?
Comp 294-9 :: Fall 2003
Interlacing• Progressive = every scan line done in turn.• Interlaced = every other scan line done.
– Creates two fields: odd field and even field.– NTSC: 262.5 lines per field at 60 fields per sec.– PAL: 312.5 lines per field at 50 fields per sec.
• Fields are separated in time.
Comp 294-9 :: Fall 2003
Analog Bandwidths• True meaning of the bandwidth.• Y, U, and V signals are all continuous along
a scan line.– A bit of a hybrid signal: discrete vertically,
continuous horizontally.• NTSC:
– Y is 4.2 MHz wide, I is 2 MHz, Q is 1 MHz• PAL:
– Y is 6 MHz wide, U is 3 MHz, V is 3 MHz
Comp 294-9 :: Fall 2003
Corresponding Data Rates• How much data can you put in 6 MHz band?
– Depends on S/N ratio.– Depends on modulation scheme.
• Typical: 27 - 36 Mbs• How many cable channels do you get?
– If 50, then 1.3 - 1.8 Gbs coming into your home.• The real question for multimedia is:
– Why haven’t we found a better use for 1.3 Gbs than continuous broadcast of Real World.
Comp 294-9 :: Fall 2003
Digital Video Frames• Almost always progressive• 3 planes of pixel values (Y, U, and V)• Pixel depth• Geometry of each plane: width x height
– Chrominance is generally subsampled.• How the planes relate to each other.• Frame rate.
Comp 294-9 :: Fall 2003
CCIR-601• Standard established for digitizing NTSC
and PAL signals.DigitalComponent
AnalogCounterpart
CCIR-601Recommendation
Height Lines NTSC: 480PAL: 576
Width Continuousvoltage changesalong scan line.
Sample at 13.5 MHzResults in 702 pixelsRecommended 720
Pixel depth Voltage range. Gamma-corrected8-bit sample.
Chrominancesubsampling
Bandwidthdifferences
4:2:2, 4:2:0, 4:1:1
Comp 294-9 :: Fall 2003
Why 8-bits for chrominance?• What’s another way to cut chrominance
bandwidth in half?– Use 4-bits per pixel.
• Why won’t that work?– Need the dynamic range for color.
• But what about when Y is either small or large?– Don’t need the range, but lots of YUV combinations
that won’t ever be used.
Comp 294-9 :: Fall 2003
4:2:2• For every 4 luminance samples, take 2
chrominance samples from odd lines and 2 from even lines.
• Chrom. planes just as tall, half as wide.• JPEG does this.
Comp 294-9 :: Fall 2003
4:2:0• 2 chrominance samples for every 4
luminance samples, odd lines only.• Chrominance halved in both directions.• MPEG generally does this.
Comp 294-9 :: Fall 2003
4:1:1• What should this be?
– 1 chrominance for every 4 luminance for both odd and even lines.
– And that is what it is.• But sometimes used to refer to this:
Comp 294-9 :: Fall 2003
De-interlacing• Since analog video fields are separated in
time by 1/2 the frame rate, at least half of a digital frame is missing no matter where you sample from.
• Normal solution: linear interpolation of even fields to produce matching samples for odd fields.
• Even better: linear interpolation of both fields to produce matching samples and thus digital frame rate will equal field rate.
Comp 294-9 :: Fall 2003
Film Frame Rates• Film is a different beast altogether.
– Continuous both vertically and horizontally– Projection is simultaneous for all parts of the
picture. – Expensive medium. – Combination of all of this motivates 24 fps.
• Film to digital is easier than video to digital.– No interlacing, sample where you want to.
• Film to video is harder.
Comp 294-9 :: Fall 2003
3:2 Pulldown• Converts 24 frames to 60 fields.
1 2 3 4 5
1e 1o 2e 2o 3e 2o 3e 3o 4e 4o 5e 5o
Comp 294-9 :: Fall 2003
Overall sampling lesson• Can’t recover what you don’t have.• Conversion between representations
requires estimation of missing samples.• Interpolation causes errors:
– Spatially: at the edges.– Temporally: when moving.
Comp 294-9 :: Fall 2003
Common Digital Video Sizes• CCIR-601 720x480 4:2:2, 4:2:0• SIF 360x240 4:2:0• CIF 360x288 4:2:0• 4:3 HDTV 1440x1152 4:2:2, 4:2:0• 9:16 HDTV 1920x1152 4:2:2, 4:2:0• 4CIF, 16CIF, QCIF
Comp 294-9 :: Fall 2003
Digital Bitrates• Current television:
– 30 fps * 720 * 480 * 1.5 * 8 = 124 Mb/s• 9:16 HDTV
– 30 fps * 1920 * 1152 * 1.5 * 8 = 796Mb/s• This motivates compression.