If you can't read please download the document
Upload
choon-keat-chew
View
21.815
Download
0
Embed Size (px)
Citation preview
A gentle introduction to video encoding
A gentle(*) introduction to video encoding
(*)for some definition of gentle
Mark Pilgrim
January 8, 2009
Video is everywhere
HandheldsiPod, cell phone, PSP, ArchosInternetYouTube, Daily Motion, MetacafeTelevisionBroadcast, cable, DVD, Blu-RayIts a birdIts a planeItsBeauregard!PCsQuickTime, Windows Media Player, VLC
Video is everywhere
But how does it get there?
Encoding process
Source material(video + audio)Raw videoCompressed videoDemuxing separates streamsEncoding compresses streamsSingle video fileMuxing combines streams again
Raw audio
Compressed audio
Encoding process
Source material(video + audio)Raw videoCompressed videoSingle video file
Raw audio
Compressed audio
Time-coded captions
Hey, dont forget about accessibility!Transcription
The Zen of video encoding
Elements of a video
The video container holds it all together4The caption format defines when captions are displayed3The audio codec defines how waveforms are compressed2The video codec defines how the pixels are compressed1
How to choose the right formats
Pick a video container that works on your target platform4Pick a caption format that works on your target platform3Pick a audio codec that works on your target platform2Pick a video codec that works on your target platform1
Constraints
CPU capacity for encoding
CPU capacity for decoding and playing in real time
Acceptable delay between recording and delivery
Acceptable quality degradation
Screen dimensions
Your bandwidth
My bandwidth
Hard limits on storage size (for physical media)
Video codec compatibility
Audio codec compatibility
Patent licensing costs
And on and on
There is no right or wrong; there is only what works and what doesnt.The Zen of video encoding
Im looking for a Citigroup quote here to use, if possible
Video codecs
The wool pulled over your eyes to blind you from the truth
A video codec defines the compression algorithm used to store the array of pixels that are eventually displayed on the screen.and they cheat. A lot. And hope you wont notice.What is a video codec?
Im looking for a Citigroup quote here to use, if possible
Acronym overload: video codecs
DiracTheoraVC-1WMVH.264MPEG-4ASPMPEG-2MPEG-1WTF?
I could tell by the pixels
Common video codecs
NoYouTube, Google VideoMPEG WGH.263HDTV broadcastFree Software hippiesBlu-RayMicrosoft world dominationYouTube HD, iTunes, Blu-RayMovie piratesDVDVCDPopular usageYes*On2, Xiph.orgTheoraNoMicrosoftVC-1Yes*BBCDiracNoMicrosoftWMVNoMPEG WGMPEG-2NoMPEG WGMPEG-1NoMPEG WGH.264NoMPEG WGMPEG-4 ASPPatent-free?InventorCodec name
(*) As far as we knowSource: Wikipedia
Video encoders
Adobe PremiereffmpegH.263nonenoneWindows Media Encoder, Compressor, SqueezeWindows Media Encoder, Expression EncoderAdobe Media Encoder, Apple Compressor, On2 Flix Pro, Sorenson Squeeze, TelestreamDivX, 3ivXCompressor, Sonic, TMPGEncQuickTimeNon-OSS toolsoggenc, ThusneldaTheoranoneVC-1dirac-research, SchroedingerDiracnoneWMVffmpeg, mpeg2encMPEG-2ffmpeg, MP1EMPEG-1x264H.264ffmpeg, XvidMPEG-4 ASPOSS toolsCodec name
Source: DigitalContentProducer.com, doom9.org, Wikipedia
Audio codecs
The hand is quicker than the ear
An audio codec defines the compression algorithm used to store the waveforms that are eventually played through your speakers.and they cheat, too.What is an audio codec?
Im looking for a Citigroup quote here to use, if possible
Acronym overload: audio codecs
DTSAC3VorbisWMAAACMP3MP2AMRSeriously,WTF?
Can you hear me now?
Common audio codecs
NoiPod, iTunes Store videosMPEG WGAACBlu-RayDVDFree Software hippiesMicrosoft world dominationMusicVCD, DVD, digital TVRingtones, VoIPPopular usageNoDTS Inc.DTSNoDolbyAC3Yes*Xiph.orgVorbisNoMPEG WGMP2No3GPPAMRNoMicrosoftWMANoMPEG WGMP3Patent-free?OwnerCodec name
(*) As far as we knowSource: Wikipedia
Audio encoders
iTunes, NeroFAACAACDTS, Inc.DolbynoneWindows Media EncoderiTunesAudio Transcoder, Easy CD-DA Extractor3GPP reference encoderNon-OSS toolsnoneDTSffmpeg, AftenAC3ffmpeg, aoTuVVorbisTwoLAMEMP2ffmpeg, RetroCodeAMRffmpegWMALAMEMP3OSS toolsCodec name
Sources: HydrogenAudio.org, Wikipedia
Captioning formats
Just shoot me
A captioning format defines when, where, and how captions for the hearing impaired are displayed on screen while a video is playing.Some formats are reused for subtitles, translations, & commentary.What is a captioning format?
Im looking for a Citigroup quote here to use, if possible
Acronym overload: caption formats
DVDMP4TTSMILSAMIASS(really)SRTTeletextLine 21BabyJesuscries
No, I cant hear you now. What else ya got?
Overview of caption formats
DVDStored as imagesDVD-ForumDVDAnalog TV (Europe)customBBC et. al.TeletextAnalog TV (US + Canada)pseudo-ASCIIElectronic Industries AllianceLine 21UTF-8(multiple)(multiple)UTF-8windows-1252EncodingWindows Media PlayerMSDN pageSAMIMP4 files, iTunes StoreQuickTime (SMIL 1.0 only)AVI files, fansubbingAVI files, fansubbingPopular usageISO 14966-17MPEG-4 Timed TextA doom9.org forum post from 2004SubRipW3C RecommendationSMILSome Word document floating around the netAdvanced SubStation AlphaSpecCaption format
Source: Wikipedia
Container formats
those should be simple, right?
A container format defines how video tracks, audio tracks, captions, and metadata are stored together in a single file.like a ZIP archiveWhat is a video container?
Im looking for a Citigroup quote here to use, if possible
Video metadata
Technical metadata
How many video tracks?
How many audio tracks?
How many caption tracks?
Where is everything within the file? (track IDs, byte ranges)
Human-readable metadata
Video title
Genre
Release date
MPAA rating
Cover art
Metadata matters
Without human-readable metadata, the iTunes Store would look like this:$9.99The.Chronicles.Of.Narnia.Prince.Caspian.2008.SciFi.Fantasy.720x480.02h29m.m4v$9.99Smart.People.2008.Comedy.720x480.01h35m.m4v$9.99Iron.Man.2008.Action.Adventure.720x480.02h42m.m4v$9.99Kung.Fu.Panda.2008.Animation.Comedy.720x480.01h50m.m4v$9.99WALL-E.2008.Animation.Action.Adventure.720x480.01h38m.m4v$9.99Step.Brothers.2008.Comedy.720x480.01h45m.m4v$9.99Wanted.2008.Action.Adventure.720x480.01h50m.m4v$9.99Hancock.2008.Action.Adventure.720x480.01h42m.m4v
Source: list of top movies from RottenTomatoes.com
Acronym overload: container formats
EVOVOBASFOGGFLVMP4MKVAVILOL
One ring to rule them all, and in the darkness bind them.
Overview of container formats
ffmpeg, MP4Box, mp4creatorMPEG-LA.MP4, .M4VMPEG-4ffmpegApple.MOVQuickTimeffmpeg, oggmuxXiph.org.OGGOggffmpeg (in progress)Microsoft.ASF, .WMVAdvanced Systems Formatffmpeg, AVIMux-GUIMicrosoft.AVIAudio/Video Interchangeffmpeg, mkvtoolnixSome guys in Russia*.MKVMatroskaffmpegAdobe.FLVFlash VideoOSS toolsOwnerExtensionContainer Name
(*) Seriously, some guys in Russia. They contributed it to the public domain, so technically you own it.Source: Wikipedia
Putting it all together
Wheres the kaboom? There was supposed to be an Earth-shattering kaboom!
Common combinations
SAMIWMAWMV, VC-1ASFWindowsKateVorbisTheoraOGGLinuxQTTextTrackAACH.264MOVMac/QuickTimeSRT, ASSMP3MPEG-4 ASPAVIDVD piratesBlu-Ray subtitles (SUP)AC3, DTSH.264, VC-1EVOBBlu-RaySRT, ASSAC3, DTSH.264MKVBlu-Ray piratesMP4MP4MP4FLVVOBContainerCustom, if anyMP3H.263Flash 8Line 21, DVD subtitlesAC3MPEG-2DVDAAC+AC3AACAACAudioMPEG-4 Timed TextH.264iPod/iPhoneMPEG-4 Timed TextFlash Timed TextCaptionsH.264Flash 9+H.264AppleTVVideoPlatform/Audience
Source: Wikipedia
Thank You!
Q&A
Google Confidential and Proprietary
Google Confidential and Proprietary
Google Confidential and Proprietary
Google Confidential and Proprietary
Google Confidential and Proprietary
Google Confidential and Proprietary
Google Confidential and Proprietary