Upload
rudolph-franklin
View
215
Download
0
Embed Size (px)
Citation preview
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
An Overview of MXF and the Search for the Video
File Wrapper.
NDIIPPPreserving Digital Public
Television
JTS 2007Toronto
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Universal Preservation Format
• Self describing storage– Must know the coding of the file.– Must know the coding of the essence
within the file.
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Universal Preservation Format
• Wrapper contains both essence and metadata– Must know the location of the essence
and metadata within the file.
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Open Source Codecs
• Report on File Formats and Packages– http://www.ptvdigitalarchive.org
• VLAN– http://www.videolan.org/
• FFMPEG– http://ffmpeg.sourceforge.net/index.php
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Source Code Links
• http://www.free-codecs.com/• http://www.avsmedia.com/downloads/
codecs.aspx• http://forum.doom9.org/• http://sourceforge.net/projects/libdv/• http://www.openquicktime.org/
– http://www.free-codecs.com/download/QuickTime_Alternative.htm
• http://dirac.sourceforge.net/• http://www.avid.com/forms/DNxHDinfo.asp
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF Background & MXF - AS/PBS
Thomas EdwardsPBS
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF – What is It?
• Material eXchange Format• Influenced by the “EBU/SMPTE Joint
Task Force for Harmonised Standards for the Exchange of Television Programmes as bit streams”
• A “wrapper format” that ties together essence and metadata elements into a coherent whole
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
What can you do with MXF?
• Store simple completed works with metadata
• Store files in a streamable format• Wrap up a playlist/EDL and essence
with synchronization information• Wrap a wide variety of compression
formats
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
KLV Coding (SMPTE 336M)
– K= Key: a unique identifier– L= Length: how long is the field
Encoded with ASN.1 BER, so can be as big as needed
– < 0x80 is actual length (0-127 decimal)– 0x8Y implies next Y bytes contain actual length– Example: 0x82.10.00, length is 0x1000 = 4096
decimal
– V= Value: what is the value of the field
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
KLV Using SMPTE MDD
06.0E.2B.34.01.01.01.02.01.05.08.00.00.00.00.00
SMPTE Designator SMPTE Item
14 47 6F 6E 65 20 57 …
Length
“Colonial House”(in ISO-7 bit code)
Value
“Version Title”
UL – SMPTE Universal Label
(16 bytes)
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Simple Versus Complex
• MXF files can be very simple– “Just wrap up my essence”– E-VTRs, Video Servers
• MXF files can be very complex– EDL’s, alternate packages– Self-referential clips– Software implementations
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Simple MXF File
• Partitions– Logical division of a file for easier parsing
• Header– Has metadata about whole file
• Body– Contains essence
• Footer– Terminates file & can have updated metadata
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Metadata in MXF• Structural Metadata
– Capability of a file & how it is constructed
• Descriptive Metadata– Editorial aspects of the file– Several “schemes” available, such as DMS-1
• Dark Metadata– Metadata unknown by some applications
• Static Metadata– Doesn’t change over entire output– e.g. Show title, producer, etc.
• Time-Dependent Metadata– Timeline or Event– e.g. “This part was shot by Bob”
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF Object Model
• Packages– A logical grouping of metadata sets
• Material Package– Represents output timeline– Owns Tracks
• File Package– Represents actual essence in the file
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Physical vs Logicalphysical - how the bytes are stored (essence container)
logical - what the bytes are intended to represent (packages)
PartitionPack
SoundElementK L fil
lK L
Hdr.
set
PictureElementK L
Hdr.
set
K L
Hdr.
set
K L
Hdr.
set
K L K L K L
Stored Picture Track
Stored Sound TrackFile Package
“played” PictureTrack“played” SoundTrack
Material Package
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF Object Model cont.
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
“Frame Wrapped”
“Clip Wrapped”
KLV Wrapping of Essence
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF Standards• SMPTE 377M MXF File Format
Specification • Essence Containers:
– SMPTE 385M SDTI-CP – SMPTE 381M MPEG (including long-GOP) – SMPTE 382M AES3 and Broadcast Wave Audio – SMPTE 383M DV 25/50 – SMPTE 386M D-10 (IMX 50 Mbps I-frame
MPEG2) – SMPTE 387M D-11 (HDCAM)
• Descriptive Metadata – SMPTE 380M MXF Descriptive Metadata
Scheme - 1 (DMS-1)
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Why Bother?
• MXF is a good way to wrap up video essence into files
• MXF is a good way to link rich metadata to video essence
• MXF has support from many vendors for interoperation (but this is a work-in-progress)
• MXF and file transfer may provide significant benefits for an IT-oriented workflow
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
PBS File Delivery Project(NGIS)
• Distribute long-form, broadcast-quality video as files
• To 180 independent public television organizations
• Five major video server vendors, and some smaller vendors
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
NGIS in Operation
Cloud
LBand
Station Equipment
NGIS Equipment
PBS
VPN
Demod Reception/Deletion/Server
4 TB
Request Automation
TrafficServerConfirmation
ArrivalNotice
TransferCommand
Monitor &Resend
OfferSchedule
FTP / CIFS
Translation(if needed)
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF Format Woes
• MXF can wrap many different types of audio and video essence
• There are a number of ways that variable size edit unit essence can be indexed
• MXF has ten different “Operational Patterns” (OPs)
• MXF can carry a wide range of descriptive metadata schemes
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF – AS/PBS
• An MXF Application Specification for PBS• Base Requirements:
– Single file per program distribution mechanism.
– Direct play back (or import) into MXF compatible servers
– Otherwise easy to translate into non-MXF compatible servers
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
AS-PBS Development
• Began in 2004 based on “AS-CNN”
• Five key server vendors to public television initially involved
• Expanded to include other server and translation vendors
• Iterative process w. teleconferences and email distribution
• SD targeted first
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF/AS-PBS Essence
• Video essence type:– MPEG-2 4:2:0
• Video bitrate:– 8 Mbps
• Audio essence type:– Uncompressed, 16-bit PCM audio in BWF
• VBI (CC/XDS):– SMPTE 334M ANC carried in SMPTE 436M for
MXF
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF/AS-PBSMXF Features
• OP1a
• Partitions no longer than 30 minutes
• Index Table segments in partition after the Essence they index
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF/AS-PBS Future
• To be defined:– HD Version– Archive version
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
MXF/AS-PBS
• Short “Golden Files” available
• Actual full-length program example to be distributed shortly (MPEG pre-charge issue being worked out)
• AS-PBS specification is available to those who want to see it, email:
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Library of Congress Stake
Carl Fleischhauer
National Digital Information Infrastructure and Preservation
Program
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Video Reformatting at the National Audio-Visual Conservation Center
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
From SMPTE 390M
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Concern for born-digital content
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Born digital examples(not moving image)
GIS
Still imageMark Yankus
Web harvest
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
http://www.ptvdigitalarchive.org/
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Preservation FormattingWorking Scorecard
• Interoperability very important• From-the-hip scorecards for
– Wrapper– Essence– Profiles/Application Specs– Metadata
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Wrapper Scorecard
• No-wrapper vs. with-wrapper• Proprietary vs. open specification• Sizing up ISO/IEC 15444-3
Motion JPEG 2000
• AAF/MXF is attractive
• How settled in the community? Moderately
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Picture Essence Scorecard
• MPEG (ITU-T) family– MPEG-2 (H.262)– MPEG-4 (H.263 but mostly H.264)– All I-frame possible (preservation value)
• JPEG 2000 family– Frame image encoding– Possible wrapper– Less adoption in still photography than some wish
• How settled in the community? Not bad, no need for one size to fit all
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Profile/App Spec Scorecard
• Important part of MPEG family from an early day– Several ISO/IEC profiles and levels– SMPTE added one for 4:2:2 @ high level
• MXF is complex– Will benefit from application specification– Good example from PBS discussed today
• JPEG 2000 family– Digital Cinema Initiative has established two profiles– LC has one for its newspaper scanning project
• How settled in the community? In the early stages
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Metadata Scorecard
• Metadata for management, support long-term preservation
• Technical characteristics– SMPTE RP-210 data dictionary (“parametrics”)– Public Broadcasting PB Core (“instantiation”)
• Preservation support– Includes preceding– Library community PREMIS project, e.g., fixity– Specifications, source code for readers/transformers
• How settled in the community? Not very
Dave MacCarnWGBH
Thomas EdwardsPBS
Carl FleischhauerLibrary of Congress
Preserving DigitalPublic Television
http://www.ptvdigitalarchive.org/