Upload
cerise
View
34
Download
0
Tags:
Embed Size (px)
DESCRIPTION
High Definition Video: Workflow from Cameras to CODECs APAN Fall Meeting Xi’an, China Aug 2007 Michael Wellings Director , Engineering Vice Chair, APAN HD WG. THINK FORWARD. THINK RESEARCHCHANNEL. Topics of Discussion. ResearchChannel Background Recent and future demonstrations - PowerPoint PPT Presentation
Citation preview
High Definition Video: Workflow from Cameras to CODECsAPAN Fall Meeting
Xi’an, China Aug 2007Michael Wellings
Director, Engineering
Vice Chair, APAN HD WG
THINK FORWARD. THINK RESEARCHCHANNEL.
Topics of Discussion
• ResearchChannel Background
• Recent and future demonstrations
• Advanced Systems
• iHDTV™ Open Source Project
• HD Video Workflow Discussion
ResearchChannel Background
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006Programs
3200
2100
1700
1270
1000
750
325
610
0
0
500
1000
1500
2000
2500
3000
3500
TotalVOD Programs
ResearchChannel Programs
More than 3,200 hours of programming in theVideo on Demand library
1,100 new programs addedthis year
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006Participants
58
45
35
18
00
0
10
20
30
40
50
60
TotalParticipants
Advanced Network Forum
AJA Video Systems Inc.
Australia's Academic Research Network
(AARNet)
Fujinon
Howard Hughes Medical Institute
IBM Corporation
Intel Corporation
Internet2
Johnson & Johnson
Library of Congress
Microsoft Research
National Academies
National Academy of Engineering
Duke University
George Mason University
Indiana University, School of Informatics
Johns Hopkins University
Massachusetts Institute of Technology
National University of Singapore
New York University
Pennsylvania State University
Rice University
Rutgers, The State University of New Jersey
Stanford University Medical Center
Texas A&M University
Tulane University, A.B. Freeman School of
Business
ResearchChannel :: Participants
Universidad de Puerto Rico
Universidade de São Paulo
University of Alaska - Fairbanks
University of Chicago
University of Hawaii
University of Maryland - College Park
University of Pennsylvania
University of Southern California
University of Virginia
University of Washington
University of Wisconsin-Madison
Virginia Tech
Yale University
National Academy of Sciences
National Institute of Nursing Research
National Institute of Standards and Technology
National Institutes of Health
National Library of Medicine
National Science Foundation
Pacific Northwest Gigapop
Poznań Supercomputing and Networking Center
R1edu.org
SURFnet
Vulcan Northwest, Inc.
Wisconsin Public Television
ResearchChannel : New Participants
American Meteorological Association
Arizona State University
California State University,Monterey Bay
Indiana University
Loma Linda University
Pacific Northwest National Laboratory
University of Kentucky
University of Michigan
Isilon Systems
U.S. Television Distribution
DISH Network satellite system 12 million homes
Cable systems 12.9 million homes
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006TV-total
24.9
21.9
18.9
18.1
12.8
8.6
7.4
0
0
0
0
5
10
15
20
25
Total Satellite and CableViewer Households (Million)
Google Video
Technology Milestones
August 1999First-Ever Streaming of High-Definition Television over the Internet
November 1999Network Sets Record Speed
April 2000First Live HDTV Over Internet Newscast
July 2004First Transmission of Full Bandwidth HD Video Between Computers
September 2005First-Ever Live HD Images from Seafloor to Land Available as IP-Based Feed
November 2005Interactive Multipoint HD Videoconference Demonstrated
November 2006Partner in World-wide testing of ConferenceXP
Technology :: Leading the Way
Continuous Ocean Observing
Consortia Partnerships
Recent and Future Demonstrations
Sept 2005 - iGrid San Diego
• iHDTV - USA118: Global N-Way Uncompressed Interactive Conferencing
• Underwater research using HD from SS Thompson – USA119: 20,000 Terabits Under the Sea
Nov 2005 - SC05 Seattle
• iHDTV /HD Storage, capture and editing using SRB (Storage Resource Broker)
Recent Demonstrations
Nov 2006 SC06 Tampa
• iHD1500 system
Dec 2006 Internet2 Fall Member Meeting Chicago
• iHD1500 system
• HD CoDecs compared
• HD Camera hands-on
Recent Demonstrations
Future Demonstrations
ResearchChannel Annual Meeting Oct07
Cinegrid / GLIF Sept07
Internet 2 Fall Member Meeting Oct07
SC07 Reno Nov07
APAN 08 Hawaii Jan08
Advanced Delivery Methods and Services
ResearchChannel Advanced on-line Services
ResearchChannel-HD
• Currently VC-1 SMPTE 421M
• AVC (h.264, MPEG-4 Part-10) coming soon
• Unicast:• http://media-wm-hd.cac.washington.edu/
ResearchChannel-HDVideo
• Multicast:• http://researchchannel.org/multicast/ResearchChannel-
HDVideo.nsc
• Solicitation for HD content
ResearchChannel Advanced on-line Services
ResearchChannel satellite feed on the network• 233.0.73.29 MPEG2@ 3.2mbps
Live 1080i HD video of Mt Rainier• 233.0.73.26 MPEG2 MP@HL @ 20mbps
Live 720p HD video of Mt Rainier
http://www.researchchannel.org/tech/desktopclients.asp
HD VOD 720p WMV-HD http://www.researchchannel.org/visions05/hdpresentation.asp
HD VOD 1080i MPEG2 ML@HL
http://www.researchchannel.org/tech/desktophdsamples.asp
KEXP-FM Audio-on-demand; uncompressed audio webcasting
http://www.kexp.org
ResearchChannel Advanced on-line Services
Developing VOD archive for oceanographic HD video
• Partnership with Isilon – to be demonstrated at SC07 Reno
KEXP-FM on-line services have a world-wide audience that equals Seattle broadcast listener numbers
ResearchChannel Advanced on-line Services
Visions ’05 - Emmy Award Nominee
First-ever live HD 1080i video/IP streamed around the world• SMPTE 292M multicast and unicast @ 1.5gbps• MPEG 2 MP@HL multicast• WMV-HD multicast and unicast
ResearchChannel Advanced on-line Services
Seattle Science Foundation Project – testbed for new
Medical technology with HD video archiving –
seattlesciencefoundation.org WWAMI: collaborative medical education via
Advanced video technology
RC HD Lab on-line
Advanced venue on UW campus commissioned
Successful launch of Experimental Internet ResearchChannel-HD
iHDTV Open Source Project
iHDTV™ Software Suite
iHDTV Explained• iHD1500
• iHD270
• HD to the Desktop
WindowsXP-based application suite
Released as Open Source
Account created on SourceForge
iHD_Trusted_Partners_Group formed to direct software development
iHDTV™ Development / Deployment
SourceForge load• SVN Tree transferred
• iHDTV_trusted_partners formed – 47 members
• 4 Core developers @ (UW / U-Mich / U-Wisconsin)
• Executive team to guide development will meet monthly
Deploy globally• N-way implementation this summer with 6
nodes
iHDTV™ iHD270
iHD270 (first introduced aug 99)
• Sony HDCamtm compression
• SDTI data format 270mbps
• AJA Xena I/O
• Requires:• AJA Xena-HD capture cards• Dual Processor P4• Windows XP• Sony HDCam Hardware encoder/decoder• Gige network connection
iHDTV™ iHD270
1920x1080 60i 8bit 4:2:2
• Sony HDCam source material
• 995mbps active lines, 1188mbps total
iHDTV™ iHD1500
iHD1500• Uncompressed SMPTE 292M 4:2:2
• Data rate total approx 1.5 gbps
• Requires:• Uncompressed HD capture cards• PCI Express platforms• Windows XP• 2 x gige or 1x10gige network connection• Jumbo frame transport and routing• Specs:
http://zzz.cac.washington.edu/ihdtv_wiki/index.php/Main_Page
iHDTV™ iHD1500
Latency about 4 frames end to end plus network delay
• 4 frames=133ms or ~ 1/7 sec
iHDTV™ iHD1500
iHD1500 Enhancements:
• New i/o board options
• GUI front end
• Self-installing package
• Uncompressed HD Server:• Single link• Dual link• Quad link
• HD Screen rendering for OptIPortal integration
• NTT i-VISTO interoperability
High Definition Workflow
A Report “in progress…”
HD – a Background
Sample Rates
NTSC 4:2:2• Y, Pb, Pr color matrix
– 4x3.38 (compromise rate from 3.58MHz) = 15.5MHz
– Color difference channels @ 6.75MHz
HD 4:2:2 (really 22:11:11)• Y, Pb, Pr color matrix
– 22x3.38 = 74.25MHz– Color difference channels @ 37.125MHz
10 bit – 1024 gradations
8 bit – 256 gradations
Sample Rates
4:2:2
• Chroma sampled at ½ rate of Luma
4:1:1
• Chroma sampled at ¼ rate of Luma
4:2:0
• Chroma sampled at ½ rate of Luma on every other line of each field
Sample / Bit Rates
Broadcast
• 4:2:0 MPEG-2 TS 19.2mbps
DVD
• 4:2:0 MPEG-2 PS VBR
HD DVD/Blu-Ray
• 4:2:0 VC-1 @ 8mbps
• 4:2:0 AVC @ 8mbps
HD Resolutions
720p 60Hz – SMPTE 274M• 1280x720 24p
• 1280x720 30p
• 1280x720 60p
1080i/p 60Hz – SMPTE 296M• 1920x1080 24p
• 1920x1080 30p
• 1920x1080 60i
• 1920x1080 60p
HD Bitrates – Uncompressed720p 10- bit 4:2:2
720p 60Hz - SMPTE 296M
• 1280x720 30p• 742mbps• 30MHz Y, 15MHz Chroma
• 1280x720 60p• 1485mbps• 30MHz Y, 15MHz Chroma
Progressive / Interlaced
1080/60i and 1080/30p occupy the same datarate – 1.485gbps
Interlaced pictures produce spacial errors associated with the time difference in motion between fields
This can create a smoothing effect which is easier on the eye during fast motion
High Framerate progressive (60 frame or greater) is best
HD Bitrates – Uncompressed10801/p 10- bit 4:2:2
1080i/p 60Hz – SMPTE 274M
• 1920x1080 30p• 1485mbps• 30MHz Y, 15MHz Chroma
• 1920x1080 60i• 1485mbps• 30MHz Y, 15MHz Chroma
• 1920x1080 60p• 2970mbps• 60MHz Y, 30MHz Chroma
Imaging chips
Solid State Camera front end (tube cameras came first!)
CCD = Charge Coupled Device
CMOS = Complementary metal oxide semiconductor
Imaging chips
CCDs collect charge at each element, and that charge is read out sequentially
Charge to Voltage conversion takes place off-chip
A>D takes place off chip
Digital level = luminance level
These are monochrome sensors!
Imaging chips
CMOS chips collect charge at each element
Charge to Voltage and A>D functions take place on-chip
CMOS imagers exhibit more noise than CCDs, and tend not to be used in higher end cameras
Single chip vs 3 chip
CCDs gather charge according to the amount of light impinging on the element
This charge is converted to a voltage representing light level
This light level does not specifically contain color information
Color reference is obtained through the use of filtering in front of the image sensor
Bayer Filter on-chip
Bayer Filter Diagram
4:2:2 sampling
Green sampled at 2x Red and Blue
Single chip color solution
Color Processing
3 chip
• RGB output from image processing
• Digital sampling and CODECS operate on component video – Y, Pb, Pr (Y’,Cb’,Cr’)
• Y = Luma channel
• Pb= subsampled blue matrix
• Pr= Subsampled red matrix
NAB 2007
HD Cameras
SONY XDCam ex
• 3 chip ½” imager (details unknown)
• MPEG-2 long GOP
• 30-50mbps
• Writes to Express Flash memory cards
• 120 min record time
• Target price $8000
HD Cameras
Grass Valley/Thomson Infinity
• 3 chip CMOS
• 2.4 million pixels each
• Ships with DV25 and JPEG2000 codecs
• MPEG2 optional
• Writes at up to 100mbps to IOMega REVPro drives
• Target price $26K without lens
• Ships in August 07
New HD Cameras at NAB 07
SONY XDCam HD
• Long GOP MPEG-2
• Writes to dual-layer BluRay Pro disk
• Up to 50mbps
• 2/3” 3 chip 2million+ pixel
• ships in 1st QTR 08
• Target price less than $30K without lens
New HD Cameras at NAB 07
Codec comparison Project
SMPTE StEM HD Mini-Movie
• Designed for critical evaluation of HD equipment as well as encoders and decoders
• Delivered as tga files, uncompressed
• Imported into Final Cut Pro
SMPTE Standard Evaluation Material
Codec comparison Project
7 Uncompressed Quicktime Movies on 7 sequences
5 movies exported:
• DVCPro 100
• HDV
• MPEG-4 Basic
• H.264 Part 10 (AVC)
• HD uncompressed
All FCP I/O via AJA Kona 2 board
Codec comparison Project
Codec comparison Project
HD sequence played out to HDCam Tape
• recaptured 8bit 4:2:2 in FCP
HD sequenced played out to NTT HD1000 MPEG-2 encoder
• recaptured from output of NTT HD1000 MPEG-2 decoder as 8bit 4:2:2, and 4:2:0
Codec comparison Project
DVCPro 100 sequence recorded to tape and played back
HDV sequence re-rendered in FCP to uncompressed
HDCAM SR recorded to tape and played back
XDCAM HD recorded to tape and played back
Codec comparison Project
MPEG-4 Basic re-rendered to uncompressed in FCP
MPEG-4 h.264 Part 10 re-rendered to uncompressed in FCP
The edit
Five scenes of interest selected
10 new sequences created for each of the 9 compressed-decompressed files plus the original
HDV and DVCPro were scaled 133 and 150% respectively to match the uncompressed material
Difference mask
Luminance of each compressed file inverted and added to the original
50% luma field indicated no loss
Material lost in codec process exhibited visually due to the subtraction process
The movies
Several HD movies resulted from this project
• 9 with the edited 5 scenes – each an example of a particular codec – showing the difference mask full screen
• 9 with the encoded and decoded video overlayed in the center of the frame of the origional for direct comparison
• 5 complete sequences of codec representation
• The original uncompressed movie
CODEC Comparison
CODECBit Rate (Mbps)
File Size (MB)
Compression Ratio Type Size Color Space Encode Time
HDCam 144 n/a 10.4:1 Hardware 1440X10803:1:1 6:30 min
HDCam SR 440 =1500/B3 Hardware 1920X10804:4:4 6:30 min
DVCProHD 115.24 5100 13:1 Software 1280x1080 4:2:2 17 min
XDCam HD 35 Hardware 6:30 min
MPEG-2 4:2:2@HL 30 n/a 56:1 Hardware 1920X10804:2:2 6:30 min
MPEG-2 MP@HL 4:2:0 18 n/a 93:1 Hardware 1920X10804:2:0 6:30 min
HDV MPEG-2 MP@H14 24.96 1110 60:1 Software 1440X10803:1:1 33 min
MPEG-4 H.264 7.572 344 198:1 Software 1920X10804:2:0 1 hr 5 min
MPEG-4 Basic 5.937 270 253:1 Software 1920X10804:2:0 24 min
VC-1 8 320 188:1 Hardware 1920X10804:2:0 28 min
JPEG2000 46/25* Hardware
JPEG2000 100/75* Hardware
JPEG2000 220/190* Hardware
equipment
SONY HDW-500 VTR
SONY HDCAM-SR VTR
SONY XDCAM –HD VTR
Panasonic DVCPRO-100
Apple Dual 2.7GHz G5• 4GB RAM• Mac OS-X 10.4.7• Final Cut Pro 5.1.1• AJA Kona 2 I/O
NTT HD 1000 MPEG Encoder
NTT HD 1000 MPEG Decoder
HD Multiburst Waveform Uncompressed Reference
HD Multiburst Waveform DVCPro 100
HD Multiburst Waveform SONY HDCam
HD Multiburst Waveform SONY HDCAM SR
HD Multiburst Waveform HDV
HD Multiburst Waveform H.264 /AVC
HD Multiburst Waveform SONY XDCAM HD 30
HD Multiburst Waveform JPEG 2K 25mbps
HD Multiburst Waveform JPEG 2K 75mbps
HD Multiburst Waveform JPEG 2K 250mbps
Seating Dist
Screens
Small screen up close vs. large screen further away
Dot pitch most important factor
Native HD resolution displays make sense for larger (24 to 50 inch) sizes due to optimum viewing distances
Low-res screens *appear* sharper from a distance – this is due to the apparent dot pitch of the digital display
High Definition Workflow
Apple FCP EDIT-SAN Architecure
Edit System Software
Apple Final Cut Pro / Final Cut Studio 2.0
OSX 10.4
XSAN 1.4
Edit Suite Hardware Systems
4 MacPro Dual Quad Core Xeon platforms
AJA Kona 3 i/o
3 XSERV
30TB XRAID
Shared storage
27.3TB shared media pool
Incremental backup of projects nightly
• IBM Tivoli Storage Manager
• StorageTek Silo archive
• Source tape is backup for media pool currently
Multi-resolution capable
HD Edit workflow
SMPTE 292M
HDW-500A
PESA 292M HD Router
Final Cut Studio edit suites
XSANXRAID Stack
X-SAN Diagram
ResearchChannel HD Migration
SD cameras to be replaced with HD cameras• Likely Sony XDCam HD• MPEG-2 4:2:2 30-50mbps IMX• High speed file dump mode from Pro Blu-Ray dual
layer disc
4:3 safe zone shooting
HD capture and edit as uncompressed HD or IMX
Can output anamorphic, 4:3, 16:9 letterbox for SD
HD project archive
Edit Room Layout
Edit Room Layout
XSERVs in Master Control
XRAID
XSAN
Studio M
Credits
NAB 2007 reports:• Noah Pitzer• Jamie Alls• Jabran Soubeih• Dave Robertson• Jack Hoffman
• Special thanks to:• Staff engineer Jabran Soubeih for CoDec
measurements• Tektronix Guide to Standard and High Definition
Digital Video Measurements– http://www.tek.com/Measurement/App_Notes/
indexes/video_audio/registration/