THE DIGITAL DILEMMA - THEODOROPOULOS

THE SCIENCE AND TECHNOLOGY COUNCILOF THE

ACADEMY OF MOTION PICTURE ARTS AND SCIENCES

THE DIGITAL DILEMMASTRATEGIC ISSUES IN ARCHIVING AND ACCESSING DIGITAL MOTION PICTURE MATERIALS

THE DIGITAL DILEMMASTRATEGIC ISSUES IN ARCHIVING AND ACCESSING DIGITAL MOTION PICTURE MATERIALS

FOREWORD

1 EXECUTIVE SUMMARY 1

2 ARCHIVING 3

2.1 History and Characteristics of Hollywood Film Archiving ............................ 52.2 Current Hollywood Film Archiving ..................................................................... 62.3 Hollywood Archives vs. Libraries ........................................................................ 7

3 THE TRANSITION TO DIGITAL 8

3.1 Audio Converts First .............................................................................................. 83.2 Visual Effects and Animation............................................................................. 103.3 Postproduction ...................................................................................................... 103.4 Exhibition ................................................................................................................ 123.5 Acquisition.............................................................................................................. 123.6 The Impact of Digital Technology on Motion Picture Archiving................ 133.7 Television................................................................................................................ 19

4 CURRENT PRACTICE • Other Industries 20

4.1 Corporate America............................................................................................... 214.1.1 Sarbanes-Oxley Act Requirements .................................................................. 214.1.2 Oil Exploration....................................................................................................... 21

4.2 Government and Public Archives in the U.S. ................................................. 224.2.1 Library of Congress.............................................................................................. 224.2.2 National Archives and Records Administration ............................................ 234.2.3 Department of Defense ...................................................................................... 23

4.3 Medical................................................................................................................... 244.4 Earth Science......................................................................................................... 264.5 Supercomputing ................................................................................................... 284.6 Summary ................................................................................................................ 29

4.6.1 Consensus View ................................................................................................... 294.6.2 Unresolved Issues ................................................................................................ 30

5 ARCHIVING IN THE CHANGING ENVIRONMENT 31

5.1 Digital Storage Technology................................................................................ 325.2 Risks and Threats to Digital Data..................................................................... 355.3 Digital Preservation Strategies.......................................................................... 38

6 DIGITAL MOTION PICTURE ARCHIVING ECONOMICS 406.1 Digital Storage Economics ................................................................................ 406.2 Digital Motion Picture Storage Economics ................................................... 426.3 What this Means for the Motion Picture Industry ....................................... 45

6.3.1 Economics of Archiving are Changing............................................................. 456.3.2 Save Everything .................................................................................................... 466.3.3 Don’t Save Everything ......................................................................................... 466.3.4 Who Decides, How, and When? ...................................................................... 46

TA

BL

EO

FC

ON

TE

NT

STable of Contents

TA

BL

EO

FC

ON

TE

NT

S

Table of Contents

7 INDUSTRY NEEDS AND OPPORTUNITIES 49

8 CONSENSUS 5 1

8.1 To Start .................................................................................................................. 5 18.2 Long Term Initiatives .......................................................................................... 548.3 Finally… ................................................................................................................ 56

9 APPENDIX – Case Study Data 58

A.1 Generic Element Trees ...................................................................................... 59A.2 Case Study Data Tables .................................................................................... 62Table A-1 – Delivered Film Capture Picture Elements ................................................ 62Table A-2 – Delivered Data Capture Picture Elements................................................ 63Table A-3 – Delivered Sound Elements............................................................................ 64Table A-4 –Storage Categories for Picture and Sound Elements

from a Film Capture Production .................................................................. 65Table A-5 –Storage Categories for Picture and Sound Elements

from a Data Capture Production ................................................................ 66Table A-6 –Estimated Annual Cost of Element Storage – Film Capture.................. 68Table A-7 – Estimated Annual Cost of Element Storage – Data Capture ................ 69Table A-8 –Estimated Annual Cost of Sound Element Storage ................................ 70

10 REFERENCES 71

11 ACKNOWLEDGMENTS 74

© 2007 Academy of Motion Picture Arts and Sciences (A.M.P.A.S). Second printing, February 2008 “Oscar,” “Academy Award,” and theOscar statuette are registered trademarks, and the Oscar statuette the copyrighted property, of A.M.P.A.S. The accuracy, completeness, and adequacy of the content herein are not guaranteed, and A.M.P.A.S. expressly disclaims all warranties, including the warranties of merchantability, fitness for a particular purpose and non-infringement. Any legal information contained herein is not legal advice, and is not a substitute for advice of an attorney.

FO

RE

WO

RD

Foreword

THE DIGITAL DILEMMA 2007

DIGITAL TECHNOLOGY IS ALREADY PRODUCING SIGNIFICANT benefits for the motion picture industry. As evidenced in image capture, visualeffects, mastering and final color grading; in sound capture, sound effects, andsound editing and mixing; and in the continually increasing digital distribution to theaters and other platforms, the digital era is not approaching – it’s here.

However, the changes have tended to arrive piecemeal, and so rapidly that theindustry has not yet had a chance to step back and consider the digital revolutionand its long-term implications as a whole. Even some of the artists who have beenthe most evangelical about the new world of digital motion pictures sometimes seem not to have thoroughly explored the question of what happens to a digital production once it leaves the theaters and begins its life as a long-term (if all goeswell) studio asset.

To date there have been no definitive studies comparing current costs of digitalor hybrid systems to those of the analog photochemical systems that have long beenthe standard in Hollywood. The long-term preservation of, and convenient accessto, a company’s cinematic assets is clearly going to be an ongoing concern, and yet a danger exists that in an effort to stay on the curl of the digital wave – an effort notsurprisingly encouraged by the vendors of digital technologies – the industry maymake decisions that produce unfortunate financial and cultural consequences.Herein lies the Digital Dilemma.

This project originated two years ago when Phil Feiner, chair of the DigitalArchival Committee of the Academy’s Science and Technology Council, proposedconvening a “summit” that would for the first time bring together archivists and senior technologists from the Hollywood studios and those charged with the preservation of moving images and recorded sound by universities, the U.S. government and other organizations. That summit led to the realization that themarked acceleration in the use of digital systems was not being accompanied byappropriate planning, or even in some cases by a full understanding of the potentialimpact of the digital revolution.

The Science and Technology Council subsequently surveyed experts in the field– from studio executives and technology department heads to those charged withpreserving medical, military and geographical data – and collected detailed informationon the issues. This report, defining the issues that the motion picture industry faceswith respect to long-term storage of and access to digital motion pictures and otherdigital assets, is the first in a series of Academy studies.

As an organization historically concerned with the art rather than the business ofmotion pictures, the Academy is appropriately concerned primarily with the culturalconsequences mentioned above. But because the business decisions that companiesmake about how to preserve their cinematic holdings, and about how much of themto preserve, have clear consequences for the art of motion pictures, this study fallsvery much within the Academy’s mission.

THE SCIENCE AND TECHNOLOGY COUNCIL

FO

RE

WO

RD

Foreword

While there have been several well-researched and informative papers on theproblems associated with digitizing existing media archives and on digital datapreservation in general – and we liberally reference some of those works here – nonehave examined the topic from the unique perspective of the Hollywood studios, aperspective that developed over a 100-year period.

From this perspective, it’s clear that a totally committed, binding switch to digital has one major drawback: the absence of guaranteed, long-term access to created moving image and sound content.

“The Digital Dilemma” is designed to bring industry executives up to date on major technological changes that are affecting and will continue to affect howcontent owners create and manage their digital motion picture materials. Thereplacement of analog film systems with digital technology has a significant impacton costs, operations, staffing and long-term access. But the motion picture industryis by no means the only one wrestling with these issues. As this report demonstrates,the federal government, the medical profession, astronomers and other scientists, themilitary and other entities are all struggling with remarkably similar issues. Throughour research, we endeavored to learn what is happening now, what problems theyhave encountered, what they foresee, and what plans, if any, they are making toaccommodate the changes that come with digital storage technologies, as well as theunintended consequences of those changes.

It is a study that offers more questions than answers. But the questions areenormous ones and they need to be addressed very soon by the motion pictureindustry as a whole, starting with those in the key corporate decision-making positions. We offer this report as a call to action to generate fruitful collaborationsand workable long-term solutions.

MILT SHEFTER, Lead, Digital Motion Picture Archival ProjectANDY MALTZ, Director, Academy Science and Technology Council

A NOTE ABOUT SOURCESMany senior and staff-level employees of the major Hollywood studios, laboratories and archive facilities spoke openly andcandidly about what is going on in their organizations and what they see happening around the industry. They also providedus with their personal views of the issues of preservation of and access to digital motion pictures. We chose to encourage thebeginning of a productive industry-wide conversation by providing a safe environment to express the unfettered views andfacts as seen by the “boots on the ground,” and in support of that openness we chose to leave this information unattributed.-Ed.

EX

EC

UT

IV

ES

UM

MA

RY

1 Executive Summary

1 / THE DIGITAL DILEMMA 2007

IN THE MOTION PICTURE INDUSTRY, THERE IS A MAJOR DIFFERENCEbetween an archive and a library. The archive holds master-level content in preservation conditions with long-term access capability. A library is a temporarystorage site, circulating its duplicated holdings on demand. An archive that storesdigital materials has long-term objectives. By current practice and definition, digitaldata storage is short-term.

For Hollywood studios, the “library,” or their collection of titles, is arguably oneof their largest and most valuable assets. For most of the last 40 years, and in manycases longer than that, they and other content owners stored all motion picture filmrecords – original camera negative through final release prints – not throwing any-thing away. The “save everything” strategy was possible because of the low cost ofstorage and long-term life of film and its supporting photochemical technology. Filmassets also served content re-purposing, even for distribution channels and marketsunknown at the time the film materials were created and saved.

In contrast, digital data practices generate much greater amounts of material, andcurrently very little of it is preserved. The digital master, created during the DigitalIntermediate process, is recorded to very stable yellow-cyan-magenta (YCM) separa-tions on black-and-white film with an expected 100-year or longer life. However, thispreserves only that singular version of the created content. The digital equivalents of“B neg,” trims and outs, and other ancillary materials available and commonly used fornon-theatrical distribution, are not saved as film but as digital data that needs to beactively managed or “migrated” to new digital media formats every few years.

The exploding use of digital technologies in acquisition, postproduction and distri-bution raises new issues related to production workflows, organizational responsibilitiesand business models. Data explosion also comes with the threat of data extinctionand, therefore, the loss of valuable content. With a single digital motion picture generating upwards of two petabytes of data – the equivalent of almost half a millionDVDs – the decisions as to what materials to hold, what to preserve and what riskmanagement decisions are needed before the migration decision, all place new pressures on management.

Current practices in other sectors such as medical, earth science, government,corporate businesses and supercomputing have spotlighted two major findings ofinterest to the motion picture industry:

1. Every enterprise has similar problems and issues with digital data preservation.

2. No enterprise yet has a long-term strategy or solution that does not require significant and ongoing capital investment and operational expense.

Experience in the above sectors underscores the fact that ongoing labor and energy costs add significantly to the total cost of ownership of digital materials.Economic models comparing long-term storage costs of film versus digital materialsshow that the annual cost of preserving film archival master material is $1,059 per title,1

1 Based on a monthly cost of 40 cents per 1,000 foot film reel in preservation conditions plus the amortized cost of film archive element manufacture.

THE SCIENCE AND TECHNOLOGY COUNCIL / 2

EX

EC

UT

IV

ES

UM

MA

RY

1 Executive Summary

and the annual cost of preserving a 4K digital master is $12,514,2 an 11-fold difference.The annual preservation costs for a complete set of digital motion picture source materialsalso are substantially higher than those for film, and all digital asset storage requires significant and perpetual spending to maintain accessibility.

Advice from the above sectors includes not allowing the equipment manufacturersand system designers to continue to foster technology obsolescence as they did in the television industry and are now doing in the information technology realm. Instead, the stakeholders must be the driving force.

There is an urgent, historically justified opportunity for content owners and archiviststo manage the transition from current to future practices. This is best accomplished whilefilm preservation can still be done in parallel, and essential digital assets that are not suit-able for film preservation are small in number and relatively young. Furthermore, the taskof preserving digital assets is too large for isolated or piecemeal efforts.

The primary challenge for proponents of digital systems is to meet or exceed thebenefits of the current film system. These benefits include worldwide standards; guaran-teed long-term access (100-year minimum) with no loss in quality; the ability to createduplicate masters to fulfill future (and unknown) distribution needs and opportunities;picture and sound quality that meets or exceeds that of original camera negative andproduction sound recording; independence from shifting technological platforms; interoperability; and immunity from escalating financial investment.

The risk management decisions about what digital materials to keep, migrate, orotherwise manage must consider the broad set of issues inherent with digital storagetechnologies. The passage of time will inevitably determine the cultural value of assets,but economics will force an ongoing assessment of the future financial value of assetseach time a major data migration is considered. The risk management decisions cannotbe postponed until the data migration deadline arrives. Digital archiving is an enter-prise-wide consideration that requires support at the highest level to be successful.

This report is a call to action for the motion picture industry to understand theissues, clearly define the problem, and create discussion among all the major stakeholdersto produce standards and technological alternatives that will guarantee long-term access ofdigitally created motion picture content. To this end, the Academy has initiated severalcollaborative projects, which include:

• research on related digital preservation issues and potential solutions• development of digital file formats for acquisition, mastering and archival applications• development of a digital preservation case study system• facilitating productive dialogue among the stakeholders

The digital dilemma arrived with the digital era. It demands concerted, committedindustry action.

2 Based on an annual cost of $500 per terabyte of fully managed storage of 3 copies of an 8.3 terabyte 4K digital master.

AR

CH

IV

IN

G2 Archiving


ARCHIVING HAS A LONG HISTORY IN HUMAN SOCIETY. KEY INSTITU-tions in every society in every era have invested in the long-term preservation ofrecords and other objects deemed important by that society at that time. From pre-history until the present age, all archives consisted of “things” that exist in the physicalworld, preserved in the physical media of each era – papyrus, parchment, paper,leather, canvas, wood, stone, ceramic, metal, silk, photographic plates, sheets and rollsof film of various gauges and specifications.

An archive is not just a collection of old content. An effective archive integratesits holdings with up-to-date catalogs, indexes and other tools needed to search andretrieve assets stored in it. Archiving purposes vary from domain to domain, and from community to community, but, in general, archiving is meant to systematically collectand protect assets considered valuable enough to keep “for the future.” Ideally, thecontents of the archive must be reliably authentic, accurate and complete. The goal ispreservation without errors, access without end.

Enter digital information: according to a 2003 study by researchers from theSchool of Information Management and Systems at the University of California,Berkeley [Lyman and Varian 1], the world generated 5 exabytes – the equivalent of5.5 trillion books – of new data in 2002, stored in four physical media: paper, film, magnetic and optical – that are seen or heard primarily through four electronic channels:telephone, radio, television and the Internet. The UCB report estimated that the 5exabytes of information recorded on storage media comprised less than one-third ofthe total volume flowing through telephone, radio, TV and the Internet in 2002.

The goal ispreservationwithouterrors, access without end.

APPROX. 21 MILLION 5,497,558,138,880

DIGITAL DATA GENERATED IN 2002, SHOWN AS THE EQUIVALENT NUMBER OF BOOKS

DIAGRAM IS NOT TO SCALE

(1 BOOK IS EQUIVALENT TO 1 MILLION BYTES)

7 TRILLION

6 TRILLION

5 TRILLION

4 TRILLION

3 TRILLION

2 TRILLION

1 TRILLION

BOOKS IN THE LIBRARY OF CONGRESS

How Much Data, 2002 – Book Equivalent


AR

CH

IV

IN

G

2 Archiving continued

“

“

“In the excitement about the solutions digitization offers, the right questions about costs are often not asked, especially about long-termcosts for keeping the digital files alive. This enthusiastic attitude is risky,for the conversion process to create the digital files may well be quiteexpensive to start with, and these investments may turn out to be wasted if planning for the future is ignored and no structural funding for maintenance is secured.

Without such long-term planning, digitization projects can come tobehave like black holes in the sky. Scanned information, which in theanalog world could be accessed simply by the use of our eyes, is suddenlystored in an environment where it is only retrievable through the use oftechnology, which constitutes a constant cost factor. The more informa-tion is converted, the more the costs for accessing it go up. The digitalblack hole has got its firm grip on the project. It will go on swallowingeither money or information: the funding must be continued or the inputwill have been wasted. If funding starts to fade, the information may still be retrieved but after a while it will no longer be accessible due tocorrupted files, or obsolete file formats or technology. Then the digitalinformation is lost forever in the black hole.”3

The problems of “data extinction” are only growing as more and more aspects ofhuman activity move to the digital domain. Consumers have to move (migrate) theirdownloaded digital music to new media players when their old players get too full,sometimes requiring re-registration of their Digital Rights Management (DRM)authorizations to insure they do not lose access to any favorite songs. Authors mustfind current applications that are interoperable with their old word processing softwarein order to read manuscripts originally written with software that has since become obsolete. Digital photographs recorded on old floppy disks cannot be accessed onmodern computers, which no longer have floppy disk drives. The only way to playold video games is to keep the old game system hardware running, which oftenrequires scouring flea markets for old circuit boards that can be cannibalized for obsolete parts. Modern digital data – the media on which it is stored, the hardwareneeded to play it and the applications that use it – are all changing at a rapid pace. In the face of these challenges, preserving digital data and assuring its accessibility over the long term requires a systematic process that is generally described as “digital archiving.”

3 From “The Digital Black Hole” by Jonas Palm, Director, Head of Department of Preservation, Riksarkivet/National Archives, Stockholm, Sweden.

Digital media are adding to the explosion of data in the world. This data explosion also comes with threat of data extinction.

AR

CH

IV

IN

G2.1 History and Characteristics of Hollywood Film Archiving


MANY CINEMA ARCHIVES AROUND THE WORLD CONTINUE TOoperate as “public” archives, such as those at the Academy of Motion Picture Arts andSciences, UCLA, the Library of Congress, the Museum of Modern Art, EastmanHouse and others. The creation of “private” archives owned by corporations for thepurpose of making money is a relatively new phenomenon in archiving history.

But in Hollywood, private film archives have emerged as valuable corporate assetsthat can appreciate in value over time and can be bought and sold for large sums.The “library” is one of the most valuable assets possessed by a studio. Assets are pre-served so they can be exploited to create new media products for future markets.Making new revenue from old assets is a very profitable approach when it can be donewithout incurring undue new costs in adapting the old media format to the new mar-ket demand.

The explicit commercial motivations of the Hollywood studio archives are amongthe factors that set Hollywood cinema archives apart from many public archives.Hollywood cinema archives are maintained by and for the content owners themselves,not by stewards holding community assets “in trust.” Another distinguishing featureof Hollywood cinema archives is the sheer size of the body of assets to be preserved,including the number of new productions that must be added to the archive everyyear to keep the collection complete. Just counting MPAA-rated films, the total number of films released in 2006 was 607, an 11% increase over 2005’s 549 films[Motion Picture Association of America 3].

While it has not always been the case, current Hollywood studio archiving policyis to “save everything,” starting with the various versions of the finished movie, butalso including all the original camera negative (OCN) film, all the original audiorecordings, all the still photographs taken on-set, all the notated scripts and more.Everything is saved from the biggest hit movies and everything is saved from the worstcommercial flops.

The modern motion picture business does a very comprehensive and reliable jobof archiving feature-length motion pictures using film archives. But looking back overthe past 100 years, Hollywood’s history of archiving has been uneven. Many of theearliest movies were lost because long-term preservation of motion pictures was notconsidered important – either commercially or culturally. Many titles in early filmlibraries on flammable nitrate stock were destroyed by fire or merely thrown in thetrash; other generations saw their film masters turn to “vinegar” in hot, humid ware-houses until current climate control requirements for long-term film preservation werewell understood. As a result, fewer than half of the feature films made before 1950have survived, and less than 20% survive from the 1920s [US, LC, NFPB, Natl. FilmPreservation Plan].

With the arrival of black-and-white TV in the 1950s, movie studios happily discovered they could generate profitable new revenues by converting old movies tovideo for broadcasters to beam to consumers in their homes. The introduction ofcolor TV in the 1960s made the color movies in the archives even more attractive for broadcasters, and more profitable for studios.

The film archive business model became even more profitable with the widespreadadoption of home entertainment packaged media, first as VHS tapes in the 1980s andthen as DVDs in the 1990s. To differentiate DVDs from VHS tapes and to justifykeeping a high consumer pricing model (despite lower per-unit manufacturing costs),the studios learned to pack DVDs with “extra value” by adding scenes, bloopers, out-takes, etc. Since it is nearly impossible to tell in advance which shots will be selectedfor inclusion in a future DVD, it has become industry practice to save all processedOCN from the original production, in addition to preserving various combinations of film separations and the interpositive (IP) of the final movie, at different physicallocations in order to protect assets by geographic separation.

The“library” is one ofthe most valuableassets possessedby a studio.


AR

CH

IV

IN

G

2.2 Current Hollywood Film Archiving

SINCE THE BEGINNING OF THE “HOME video era” around 1980, most studios have come to recognize the potential long-term value of their filmlibraries and some have embarked on ambitious “assetprotection” programs. Paramount Pictures is a case inpoint. From 1987 to 1993, Paramount reportedly spentover $35 million inspecting its negatives, audio tracksand black-and-white separations; doing film repairs; andprinting new preservation materials. In 1990 it opened anew $11 million archives building, with environmentallycontrolled vaults for preprint and color materials.Paramount stores some of the master elements in anunderground facility in Pennsylvania and tracks millionsof items worldwide through a custom-designed comput-erized inventory and tracking system. By investing inthe physical care of its collections, the studio expects to extend the shelf life and revenue potential of filmelements as well as to expedite retrieval. A similararchive construction and asset-protection project wasundertaken by Warner Bros.

Industry storage practices vary, of course. Otherstudios have their own film vaults on their premises, andstore other film material at commercial vaults. In addi-tion, most large studios routinely keep preservation mastersof films they produce as well as additional materials –such as foreign-language soundtracks or edited televisionand airline versions – as required for ancillary markets.For each title, a studio may keep many differentpreprint and sound elements. The depth of preserva-tion protection depends on the scope and duration ofthe studio’s commercial rights and the film’s expectedvalue over time.

For most people in Hollywood today, the terms“preservation masters” and “archival masters” describethe 35mm OCN, IP used in the print manufacturingprocess, and YCM separations on black-and-white filmstock stored in environmentally secure film vaults. TheOCN is the most fragile and is only accessed to makenew IPs or restoration elements when needed. The IP,usually extensively duplicated, is typically used to makenew printing masters, and the separations are used whenall else fails.

Each studio has its own list of what specifically

should be archived. But generally speaking, the issues of analog archiving are well understood by many people. After more than one hundred years of technicalinnovation and market forces, the photochemical mediaformats have settled down to just a few remaining choices, manufactured by only a few remaining vendorsin widely accepted standard formats. Today, there isbroad consensus about how to manufacture and preserve35mm film archival masters.

One of the largest film vaults in Hollywood, as anexample, holds about 425,000 film elements of variouskinds. This particular vault holds film elements frommotion pictures produced in 1912. Like all theHollywood film vaults built (or renovated) in the lastfifteen years, this facility is designed for preservationstorage with cold temperatures, low humidity and firesuppression systems.

Archive services are basically the same for whateverfilm element is placed in the vault, whether it is a television program, theatrical release, or documentary,no matter if it is cut or uncut. Every piece of filmcoming into the vault is inspected before storage.Inspectors manually inspect the film, look at the information on the leader and compare it to the container labeling. In addition to inspecting the filmelement for physical and photographic integrity, a staffrestoration management director may have a laboratorymake a viewing print to ensure that there is nothingwrong with the master negative, and that it is intactfrom first frame to last before committing it toarchival preservation.

As part of the intake process for every film element, the archive staff manually logs basic asset management information such as element title, reelinformation, element type (OCN, IP, etc.), versiondescription (editor’s cut, etc.), program type (theatrical,TV, cartoon, documentary) and aspect ratio. Also, oneor more unique barcode identifiers are assigned forinventory management. For a new release today, bythe time the original lab materials reach the vault, aHigh Definition (HD) master has already been made,and this is used to supply various downstream electronicdistribution platforms.

The terms “preservation masters” and “archival masters”describe the 35mm original camera negative (OCN), interpositive(IP) and yellow-cyan-magenta (YCM) separations on black-and-white film stock stored in environmentally secure film vaults.

AR

CH

IV

IN

G2.3 Hollywood Archives vs. Libraries


FILM ARCHIVES IN DISTANT, COLD-TEMPER-ature underground vaults today are accessed only whennecessary – for example, if no other good-quality filmprint master elements can be found locally. Sometimesthis means an entire movie must be retrieved; sometimesjust some short elements are needed to repair or replace aparticular scene. They function as a kind of insurancepolicy to protect valuable assets produced at greatexpense. These are carried on the financial books ofglobal media companies who have, over the years,bought and sold their film collections for millions oreven billions of dollars.

In parallel to film archives intended for long-termpreservation, studios operate short-term film distribu-tion libraries containing release prints, interpositiveand/or internegative film copies that can be used tomanufacture new release prints and other finished elements (including sound tracks) needed to meet commercial distribution requirements. The assetsstored in the distribution libraries are accessed frequentlyand are very actively managed to satisfy customerdemands and maximize revenue potential during theprimary commercial window for each title produced,typically a period of three to five years.

While the major studios’ archives have stayedalmost 100% film-based so far, the commercial distribution libraries operated by these same studioshave expanded in recent years to include not only filmprints, but also Digital Cinema Packages (DCP) as wellas derivative versions of programs in digital formats fornon-theatrical release, such as Standard Definition andHigh Definition video for sale to television broadcastersand cable and satellite system operators. Several peopleinterviewed for this report believe that as higher per-centages of a studio’s revenue potential for a given titlecome through non-theatrical digital markets in new formats, there will be growing pressure to move the distribution libraries to digital platforms to stay com-petitive. At least two major studios, Sony Pictures andWarner Bros., have digital distribution library projectsunderway: ATLAS (through Ascent Media Group) andDETE (Digital End-To-End), respectively.

The traditional analog system that separates

archives for asset preservation from libraries for distri-bution is being carried over to the digital domain. The digital media archival assets are most likely full pixelcount, full bit-depth, uncompressed and unencrypted,as compared to the digital media distribution librarycontent, which is most likely formatted at lower pixel count, lower bit-depth, and compressed.

Titles in the distribution library might be pre-encrypted, ready to go on demand. Or they might be stored un-encrypted inside the library, but alwaysencrypted as part of the distribution process. Titles inthe library might also contain embedded watermarkingand other DRM metadata.

Digital archives are only truly protected by redundantreplicas of the structured digital assets themselves. Newtitles move into the distribution library faster than theyare added to the archive because the distribution library isused to generate revenue while the archive is intended toact as insurance against any loss of corporate assets. But ifdigital motion pictures can become “born archival,” theycan be ingested into the archive quickly and easily as partof a largely automated file-transfer process.

The storage and administrative systems for the digital preservation archives and for the digital distribu-tion library might well merge into one unified repository,perhaps employing different user interfaces – one forlibrary services, the other for archive services. Archivalassets would typically require reformatting when theyare retrieved as a library service, but not when they areaccessed through the archival interface.

The conversion of archival formats into distributionformats has historically required slow and/or expensiveprocessing, often using purpose-built hardware forspeed. But continuing increases in digital processingpower, digital storage capacity and digital networking bandwidth mean that it may become more efficient to co-locate the archive, library and distribution infrastructure. This could reduce the number of datatransfers from archiving to processing facilities and consolidate many redundant functions shared bylibraries and archives. It also would bring together thepeople responsible for preservation and those working on distribution and media processing R&D.

Digital archives are only truly protected by redundant replicas of the structured digital assets themselves.


TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L

3 The Transition to Digital

IT IS IMPORTANT TO UNDERSTAND THAT THE MOTION PICTURE industry has been adopting digital technologies in a piecemeal fashion over the last 25 years. The following sections present a brief history of the digital conversion. The recent introduction of digital technologies into the final links in the productionand distribution chain is, in fact, a “tipping point” that fundamentally changes theindustry’s economics and practices.

The digital transition affected different aspects of the production process at different times, although the fully digital production still results in a “film-out,” orcreation of a film negative from the final digital master. This, along with the YCMblack-and-white separations made from the final digital master, is the only finishedfilm asset that is currently being saved using a well-recognized technology with understood and accepted long-term preservation and access characteristics.

3.1 Audio Converts First

MODERN AUDIO RECORDING, POSTPRODUCTION AND DISTRIBUTIONall use fully digital workflows yielding digital audio files best saved on digital storagemedia. In fact, analog audio tape is rapidly disappearing. There are few remainingmanufacturers of analog audio tape or the professional recorder/player devices that canhandle such tape. This is compelling the sound departments of the major Hollywoodstudios and elsewhere to transition to digital archiving for lack of a better alternative.

Digital Audio in Acquisition and PostproductionThe introduction of digital audio recorders and processing equipment in the early1980s was the start of the motion picture industry’s conversion from analog electronicsand film technology. The Nagra series of analog audio tape recorders manufacturedby the Swiss company Kudelski, S.A., long the de facto standard for motion pictureproduction sound recording, began to be replaced by the Digital Audio Tape (DAT)format, which was subsequently replaced by field recorders that use hard drives andrecordable optical storage devices. By the late 1980s, the supporting analog mixingconsoles and tape recorders used downstream for sound editing, effects, and mixingbegan to be replaced by Digital Audio Workstations (DAW), although the finalsoundtrack continues to be output in analog form to film stock coated with a magneticlayer (“fullcoat mag”), and then ultimately as an analog optical track on film prints.

Digital Audio in ExhibitionAlthough it was announced in late 1990, it wasn’t until 1992 that Dolby Laboratoriesfirst introduced the SR/D format, known today as Dolby Digital, with the release ofBATMAN RETURNS. The development that made this format possible was the AC3audio data compression algorithm for 5.1 audio channels, with the “.1” signifying a limited-frequency bandwidth subwoofer channel. Real estate on film is precious, andso Dolby opted to record the “bit map,” or images representing the actual digital bits,between the sprocket holes. It should be noted that the optical analog sound trackwas retained as a backup measure, which still remains on film prints today.4

More digital formats then appeared in the cinema marketplace. Digital TheaterSystems (DTS) introduced the DTS digital 5.1 format in 1993 with the release of

The recentintroductionof digitaltechnologiesinto the finallinks in theproductionand distributionchain is, in fact, a “tippingpoint” thatfundamentallychanges theindustry’seconomicsand practices.

4 Optical Radiation Corporation was the first producer of a commercial theatrical digital audio reproduction system, used first on DICK TRACY in 1990, but the lack of an optical backup track, coupled with the system’s complexity, prevented the system from being adopted by the major studios.

TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L3.1 Audio Converts First continued


JURASSIC PARK. DTS places the digital audio bits on CD-ROMs in a proprietary format and records only an analog synchronization track on the film, also preservingthe analog optical track as a backup.

In 1993 Sony introduced the SDDS digital audio format with the dual releases ofIN THE LINE OF FIRE and LAST ACTION HERO. Unlike Dolby Digital and DTS,SDDS is a 7.1 format, resurrecting the additional full-range effects channels of theTodd-AO 70mm magnetic format, although not all feature films are released usingthis capability. As with Dolby Digital, the SDDS data is recorded directly on thefilm, and as with both of the other digital formats, SDDS relies on the stereo optical track for backup [Karagosian].

It is important to note that each of the existing digital formats occupies an exclusive physical area of the film. In practice, it is more and more common to releasea film print with the printed digital audio data or time code for more than one format.Film producers enjoy the choice and innovation that come along with multiple com-petitors in the marketplace. There are limitations and advantages to each of the formats, in terms of sonic capabilities, distribution capability, and the economics ofthe film print itself. For the foreseeable future, there will continue to be a variety offormats for multi-channel sound for cinema.

Archiving Digital AudioStudio sound and preservation departments have long known that digital audio tapeformats do not have adequate long-term survival characteristics, primarily due to their“brick wall” failure mode. That is, while analog audio tape degradation manifests asincreased audio “noise” which can generally be filtered out, digital audio tape degradation manifests in the inability to recover any of the sound at all. It is for thisreason that some studios have backed up their digital audio data to recordable CDswith scheduled migration to recordable DVDs. However, according to the NationalInstitute of Standards and Technology, DVD technology has degradation characteristicssuch that approximately half of a collection of disks can be expected to last more than15 years, and therefore half will not [The X Lab].

Digital audio preservation methods are getting more sophisticated. In a presentation at the Association of Moving Image Archivists’ May 2007 Digital AssetSymposium, NBC/Universal Studios discussed the development of its digital deliveryand preservation system that uses a combination of online hard drives, LTO3 datatape, and DVD-R optical disks to access and preserve its motion picture sound elements [Taylor and Regal].

Digital audio tapedegradationmanifests inthe inabilityto recoverany of thesound at all.


TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L

3.2 Visual Effects and Animation

Digital AssetManagementsystemsrequire ongoinginvestment ininfrastructure,hardware,software, andhighly trainedpersonnel.

JURASSIC PARK WAS NOT ONLY A WATERSHED EVENT FOR MOTIONpicture sound; it is also widely regarded as the first major motion picture to use photore-alistic digitally created characters in a central role.5 The movie’s dinosaurs were originallyplanned to be shot with traditional stop-frame animation techniques using miniaturemodels, but the initial tests of the digital dinosaurs were so promising that a commitmentwas made to go completely digital. The final product was impressive, and the popularfolklore is that audiences “could not tell the digital dinosaurs from the real ones.”

1995’s TOY STORY was the first feature film with completely computer-rendered 3D characters, and in the years since, 2D animation and visual effects have been almostcompletely created using digital tools.

The adoption of purely digital tools for visual effects and animation created a needfor effective digital data management tools for production activities, also known asDigital Asset Management systems (DAMs). DAMs, in most cases, effectively enablethe backup and production-related access of the digital character models. This is notwithout its costs, as they require significant investment in Information Technology (IT)infrastructure, ongoing hardware and software upgrades, and highly trained personnel.But the combination of digital visual effects and DAM has proven effective in makingsome of the most commercially successful movies of the past few years.

3.3 Postproduction

Picture EditingThe transition from cut-and-splice film editing to electronic nonlinear editing began inthe mid-1980s with the introduction of computerized videotape- and videodisc-basedediting systems. Film-originated television programs were the first to adopt these systems because they did not require the conforming of the film negative to produce the final edited master. Television program masters were assembled from master videotapes using electronic “instructions” generated by the nonlinear editing systems.The development of “negative cut lists,” coupled with the instantaneous access of digitalvideo stored on computer hard disks in the early 1990s, made electronic nonlinear editing practical for the editing of feature-length motion pictures.

Today, almost every theatrical motion picture is edited on a digital nonlinear editingsystem, and consumer versions of this professional tool have found their way into tens ofmillions of homes. For better or worse, the rise of personal video-sharing websites suchas YouTube would not have happened without the development of professional digitalvideo editing tools.

It should be noted that in the three cases discussed to this point, the transition tocomplete adoption of each of the digital technologies took no more than ten years frominitial commercial introduction.

MasteringThe final step in motion picture production is in the midst of its conversion from filmto digital technology. Generally referred to as the Digital Intermediate process (althoughDigital Mastering is a more appropriate term), the final color balancing and visualstyling of the final master film record more often than not are done using digital toolssuch as interactive color correction systems rather than adjusting film stock exposureand developing controls. The Kodak Cineon system, introduced in 1992, demonstratedthat analog film images could be converted to digital bits, processed and enhanced, andthen re-recorded back to film with powerful results. This concept is used for both visual

Interchange of imagesbetween facilities, arequirement in today’sworld of multi-facilitycollaboration,is problematic,given the lackof digital file format standards.

5 Earlier motion pictures such as 1985’s YOUNG SHERLOCK HOLMES and 1989’s THE ABYSS integrated computer-generated characters, but in relatively small supporting roles.

TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L3.3 Postproduction continued


HDTV DIGITAL CINEMA DIGITAL CINEMA DIGITAL CINEMA FILM

Pixel Count

Color Gamut

Precision

1920 X 1080 2K 4K 35MM

1920H X 1080V 1920H X 1080V 2048H X 1080V 4096H X 2160V ˜4096H X 2160V*

FORMAT

DIAGRAM IS NOT TO SCALE * Approximate pixel count of 35mm film negative

effects integration and final color balancing, and it is widelybelieved that more than half of all major motion picturestoday are mastered using the Digital Intermediate process.

As with any newly adopted process, there are unre-solved issues. For example, some mastering facilities useHigh Definition Television (HDTV) equipment ratherthan higher-quality “4K” computer-based systems6 as a cost-saving measure. The resulting master contains less visualinformation in terms of fine detail and dynamic range (collectively referred to as “precision” in the diagrambelow), is of observably lower image quality than what hasbeen achieved for over 100 years with film, and there isconcern that the decision to archive reduced-quality masterswill have adverse consequences in the future [Scherzer].Interchange of images between facilities, a requirement

in today’s world of multi-facility collaboration, is prob-lematic, given the lack of digital file format standards.7Furthermore, the final physical form of the digital master – data tape, optical disk, magnetic hard drive –is not defined by any standard or industry agreement,and therefore what goes into the archive is not defined.

Another unintended side effect of the DigitalMastering process is that the final digital master inmany cases bears little resemblance to the original camera negative (or digital camera original data if a digital camera is used in production). The level of creative control enabled by digital mastering tools effectively shifts significant decisions regarding themotion picture’s overall “look” downstream from on-setchoices historically made by the cinematographer.

Visual Attributes of Image Formats

6 “4K” is shorthand for the highest pixel-count digital motion picture image format in regular use today. A 4K image has 4096 pixels in the horizontal direction and 2160 pixels in the vertical direction, which is roughly equivalent to 35mm film.

7 The Academy has a project underway to address the interchange issue. More information on this project can be found at http://www.oscars.org/council/advanced.html.


TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L

3.4 Exhibition

THE CONVERSION FROM FILM PROJECTION TO DIGITAL CINEMAprojection technology8 is underway, and so much has been written and is still beingwritten about this topic that the subject will not be covered here except to point outthat it is unclear at what time in the future film prints will become obsolete. Of theapproximately 37,000 commercial theaters in the United States, 3,595 are DigitalCinema-enabled, and conversions are occurring at the rate of approximately 200 screensper month [DCinema Today; Overfelt]. The conversion rate is expected to acceleratewhen Digital Cinema Implementation Partners, a consortium representing over 14,000U.S. screens, and Technicolor Digital Cinema begin their deployments. Assuming adoubling of the current conversion rate to 400 screens per month beginning in 2008,there would still be over 8,000 film-only screens remaining in the U.S. in 2013. Theinternational conversion rate is expected to be slower than the domestic rate, given theunique business and governmental issues. With over 70,000 screens outside the U.S.,there are likely to remain a substantial number of film-only screens for some time. Atwhat point film prints are no longer economically justifiable is unknown.

3.5 Acquisition

DIGITAL MOTION PICTURE CAMERAS THAT IN SOME RESPECTS MEETor exceed the perceived image quality of 35mm film negative are now in regular commercial use. The digital output of these cameras is recorded either to HDCAM SRdigital videotape, a magnetic hard drive – based digital recorder or solid-state “flash”memory devices. According to the motion picture camera manufacturers interviewedfor this report, approximately 20 to 30 major motion pictures per year are now shotusing these cameras. Reported advantages of these cameras over film include immediateplayback of recorded scenes in certain circumstances, increased color saturation in lowlight-level situations, and longer recording duration between media reloading.Reported disadvantages include reduced spatial resolution and exposure latitude relative to 35mm film and postproduction workflow challenges caused by the largeamounts of digital data produced. Additional trade-offs must also be considered whenchoosing the capture medium for these cameras: HDCAM SR digital videotape or digital data recorders. For example, HDCAM SR uses mild image compression and digital data recorders do not; digital data recorders allow for deferring certainimage processing choices until later in the postproduction process; and higher spatialresolutions and greater bit-depths are possible with digital data recorders.

This new capture technology has had some interesting effects on production practices. For example, the relatively low cost of digital videotape as compared to film negative has resulted in letting the camera roll for much longer periods of time thanwith film, enabling directors and actors to spend more time achieving a desired per-formance [Kirsner]. There is some concern that the greater amounts of source materialgenerated with this production style will result in added overall costs when postproduc-tion and archiving costs are factored in. It is also reported that some directors will do“circle take” selection on-set, deleting the digital files containing takes they know theywill not use [Hurwitz]. The concern voiced about this approach is that it increases therisk of accidentally erasing a good take or eliminating potentially useful alternate takes.

There is, of course, the matter of how (or whether) to preserve the enormousamount of digital data produced when shooting digitally. This subject will be delved

At what point filmprints are no longer economicallyjustifiable isunknown.

There is, ofcourse, thematter of how (orwhether) topreserve theenormousamount ofdigital dataproducedwhen shooting digitally.

8 “Digital Cinema” is defined as the theatrical projection standards currently being developed by the Society of Motion Picture and Television Engineers’ DC28 Technology Committee.

TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L3.5 Acquisition continued


into in Section 6, though it is worth mentioning here that one of the motion picturesanalyzed for this report produced well over 5,000 HDCAM SR videotapes from location shooting. Digital acquisition using uncompressed digital recording systemssuch as a magnetic hard drive recorder or a solid-state “flash” memory recorder generates 60 to more than 2,000 terabytes of data (depending on pixel count, bit-depth, number of backup copies, etc.) or the equivalent of 13,000 to 436,000 DVDs.By any measure, this is a large number of tapes or disks to consider when archivingoriginal source material.

What is unknown at this time is whether digital cameras will ultimately supplant35mm film as the primary capture medium for theatrical motion pictures. Cinema-tographers using the new digital cameras seem to agree that they are simply anothertool in their creative toolbox, and that shooting with film still has its benefits. As withprint film, it is unknown at this time whether the economic viability of film negativewill be diminished as a result of the adoption of the new digital tools.

3.6 The Impact of Digital Technology on Motion Picture Archiving

THE ADVENT OF DIGITAL CINEMATOGRAPHY, WIDESPREAD ADOPTIONof Digital Mastering postproduction workflows, and the studios’ push to deploy digitalcinema distribution to theaters means that the cinema industry must reconsider itsexclusive dependence on “film in a cold room” for long-term preservation of its motionpicture assets. Studio representatives readily acknowledged in interviews that they seean emerging need to archive their digital assets, which are growing in number and varietyand potential value. Traditional film archiving can no longer preserve all the forms ofoutputs flowing from the creative processes at the heart of the studio’s business.

Generically, digital archiving is the systematic digital ingestion, storage, preserva-tion and access, with the intention of long-term preservation, of digital “objects” comprising structured data files in a format that can be indexed and searched in somemanner. When it comes to cinema, digital objects commonly include sequences of digital image frames that make up digital masters, multiple digital sound tracks, foreign-language dialog tracks, and text files containing subtitles in various languages.They may also include digital camera originals, digital audio original stem files, pre-mix/pre-dub audio files, and other digital “assets.”

According to a 2005 paper from Stanford University researchers [Rosenthal, et al. 1],the goal of a digital preservation system is that the information it contains remainaccessible to users over a long period of time. The key problem in the design of suchsystems is that the period during which such assets need to be accessible is very long –much longer than the lifetime of individual storage media, hardware and softwarecomponents, and the formats in which the information is encoded. If the period wereshorter, it would be simple to satisfy the requirement by storing the information onsuitably long-lived media embedded in a system of similarly long-lived hardware andsoftware. But today, no media, hardware or software exists that can reasonably assurelong-term accessibility to digital assets. A dynamic approach that anticipates failuresand obsolescence will be essential.

Archiving of digital assets is a new challenge for the studios. At one studio, thereis a huge backlog of films waiting to be ingested into a digital archive. All digital elements are going into “temporary digital storage” where they will not be looked atagain until they need to be migrated some years from now.

Today, nomedia, hardware or software exists that can reasonablyassure long-termaccessibility to digitalassets.


TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L

3.6 The Impact of Digital Technology on Motion Picture Archiving continued

The discussions with studio personnel were wide-rangingand in depth on the challenges they face in this changingenvironment. In general, there is no clear picture ofhow to deal with not only the production and intermediatedigital elements, but also the proliferation of differentfinished versions of a movie (e.g., foreign language versions, edited for cultural sensitivities in other mar-kets, etc.). There is also much concern about the trendto create digital masters at 2K (only slightly better imagequality than HDTV), which contains significantly lessvisual information than the film masters created today, oreven those created 40 years ago. The fear is that projec-tion and display technology will continue to improve, butthe archived source material will produce nothing betterthan what can be seen on today’s display technologies.

On the topic of storage media, LTO3 data tape isbeing used at one studio as an archival medium becausethey believe there is no better choice, and they recognizethat this commits them to migrating the data to a newerformat sooner or later. They also recognize that therehas been no planning for that eventuality. The biggestchallenge is that they are worried about making thewrong choice, given that their long-term objective isone hundred years of content life with guaranteedaccess. They also believe YCM separations are the bestprotection and insurance available today, because theyprovide a safety net to allow the use of a digital storageformat with a much shorter lifespan such as LTO3, andto find a better solution within 7 to 10 years, assumingLTO3 lasts that long. They emphasized that digitalarchiving of the finished program is the number onepriority. Archiving so-called “floor content” (trims andouts) that are saved today as part of the film archivingsystem is a secondary concern for this studio.

At another studio, digital acquisition of principalphotography for “A titles” is seen as a looming chal-lenge. This studio already saves key components,including the digital output negative and separations,but they do not have a system to save the original digital camera data. Ultimately, they want a method forlong-term digital storage that works as well as film does.They are confident that it is feasible to keep digital elements protected and accessible for 5 and possibly upto 10 years, but long-term digital archiving is anunsolved problem. They are hoping for help from thestorage industry in terms of new archival-quality mediaand other non-film archival methodologies that can beapplied to digital sound and to digitally capturedmotion pictures. Eventually, if they are no longer ableto output to film, then of course everything will need to go to digital storage. There is concern about the economics of digital archiving, but the bigger fear is

that the studio will not adequately invest in future-proof archiving and access, and will thereby risk thelong-term survival of expensive corporate assets that alsohave important cultural value.

At a third studio the most immediate problem isagain how to handle digital camera origination materialsstored on hard drives. Some digital cameras are usingdigital videotape, but many are recording straight tohard drive or flash memory storage. Without physicalcapture media such as tape or optical disk, there is noeasy way to keep the entire digital negative. Studioarchivists do not know if they have archived everythingbecause no physical media exist – there is no equivalentof OCN in these cases. They are also encountering thisproblem with audio recordings, and comment that it isunlikely that every digital videotape or digital audio tapein storage today can be transferred to data tape in thefuture because of cost. Management of metadata – the“data about the data” that allows for efficient indexing,search and retrieval – is critical for archivists but is not ahigh priority for manufacturers or users.

At still another studio, senior technologists worrythat there is no formal corporate archiving strategyacross the whole company. The technology group canlead (intellectually) and formulate “recommended prac-tices” that the business units may adopt or not, as theywish. Strategy decisions for archiving are largely madeat a business unit level. The business unit that is taskedwith storing assets has to pay the storage costs. Archivingissues are currently handled in a decentralized manner,but people are starting to realize that if the differentbusiness units were to compare notes and start to shareresources even a little bit, the enterprise as a wholecould be more efficient about how to tackle the digitalarchiving challenges. Given the complexity of internalaccounting, operational and business responsibilities,the best approach, they think, would be to centralizeknowledge about digital archiving but decentralize control and budgeting for specific archiving facilitiesand the assets they hold. Stand-alone “silos” of digitalarchives remain, and basic problems related to internalarchiving are unresolved. Therefore, compatibility withexternal archives is still a low priority.

One executive argues that digital archiving is strategicto the future global media business of the studio, andthat this work needs to be done fairly close to homebecause the production processes and the archivingprocesses are getting interwoven. There is an assump-tion that everything produced in the future is going toget re-purposed, sliced and diced in many different waysfor different markets over many years. “It’s not like thefilm vault of old where you could ship stuff off to

TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L3.6 The Impact of Digital Technology on

Motion Picture Archiving continued


0.00.20.40.60.81.0

CYAN

OPTICAL OR DIGITAL RECOMBINE

LASERRECORDER

RECORDING FORMAT: 4K OR GREATER

Making a Digital YCM Separation Archival MasterOn Black-and-White Polyester Film Stock

CHECK PRINTDUPLICATENEGATIVE

MAGENTA

YELLOW

underground mines used for storage, and then call a few times a year and tell them to ship stuff back toyou.” The assets in the archive all need to be treateddynamically now.

One senior technologist anticipates that the biggestchallenges for digital archiving in a “studio culture” willbe organizational, requiring long-term educationalefforts, process re-engineering and self-discipline.

At another studio, a senior technology executiveexplained that, ideally, he would like to have all hisarchived assets available on online magnetic hard drivesso that in 50 years the studio can use computing powerto do things no one has thought of today. This wouldenable new video and audio search tools to automate thecataloging/metadata bottleneck. He believes that the stu-dio ultimately wants instant accessibility to everything.

Another studio executive explained that in addition

to wanting to archive new HDCAM SR tapes as originalfootage from digital cameras, his company wants toarchive all of the scripts and the shooting logs. Buteverything on paper goes into the “all paper” archivestorage facility, while anything to be saved from photoshoots is sent to a different library than the videotapesthemselves. The hope is that everything will eventuallybe connected through the use of metadata and databases.But today there are still a lot of cardboard cartons justfilled with un-inventoried “stuff.”

On the other hand, a very senior studio executiveexplained that archival storage of major motion picturesis both a financial obligation and a cultural obligation.He feels a heavy responsibility to protect the studio’songoing archival preservation activities against all risks,internal or external. In his view, this makes it imperativethat any strategy for archival preservation be able to

Making a Digital YCM Separation Archival MasterOn Black-and-White Polyester Film Stock


TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L


survive even the potential risks of global economicbad times or investor-driven corporate budget cutsfrom “on high.” He emphasized the criticalrequirement that truly archival assets be able tosurvive even if someday there is no money for thenext data migration. For him, “archival” means“store and ignore,” with the belief that survival for20+ years without worry, and playability 50 to 100 years from now without major additionalinvestment and in the face of “benign neglect” are essential ingredients of any sustainable archivingstrategy. The problem, he believes, is that nomodern corporate institution will or can commit to“forever” funding. His studio today also holdsdata tapes, but does not consider them archival.They would like to be able to archive data – treatit as an archival asset – if and when there are goodsolutions to the problems of data migration andsimilar digital preservation strategies. The onlything that meets this studio’s definition of“archival” now is 35mm film, so “archival preser-vation” at this studio depends on film prints andpositive/negative elements, plus YCM black-and-white separations on film that go into deep(underground) storage. His company’s film IPtypically goes bad every 6 to 7 years due to repeateduse. If the OCN is in good shape when the IPgoes bad, the IP can be remade from the OCN. If not, it can be reconstructed from the black-and-white separations. If digital files need to be reconstructed in the future, the black-and-whiteseparations can be re-scanned using the higher-quality, faster digital scanners of the future. Hewould like to find a digital alternative to filmarchiving, but has not seen it yet. And until then,the only thing the studio can truly depend upon isfilm for archival preservation.

Hollywood will probably continue to archivenew motion pictures on film as long as film stockand film processing remain available and economical.The simplicity and dependability of film’s “directview” access compared to the software-based“interpreted view” of digital content continues to be attractive to many in Hollywood. The economics of film archiving are well understoodcompared to those of digital archiving, and film archiving requires little new investment.Furthermore, old motion pictures already in thefilm archives are expected to survive intact for thenext 50 to 100 years, assuming the temperatureand humidity in the film vaults are maintainedunder proper film preservation conditions. If forno other reasons, institutional inertia and the

natural conservatism of studio management will tend to extend the use of film for archiving of motion pictures. Interestingly, the catalogingand indexing systems for film archives, especiallythe crucial metadata databases needed to imple-ment any enterprise-wide DAM system, havealready gone fully digital in many cases, althoughthere is no commonality of implementationamong the studios.

Cinema motion picture archiving mustencompass digital archiving; “born digital”assets have no film elements to preserveLike all the other media industries that haveadopted digital technologies before it, the cinemaindustry is starting to generate an increasing percentage of important media assets that have no analog version – that is, they are not created onfilm in the first place. These assets are “born digital.” The growing use of digital cameras forprincipal motion picture photography on “A title”studio movies means that instead of original cameranegative at the end of a day’s shooting, there areboxes of HDCAM SR videotapes or terabytes ofcamera-generated data files on disk-packs and datatapes. The move away from shooting film is alsoassociated with a reported trend to higher shootingratios, yielding more boxes of videotape or moreterabytes of data, depending on the production.The output of the Digital Mastering process is nolonger a cut negative in many cases, but rather ter-abytes of uncompressed digital frames on magneticdata tape. And with the spreading deployment ofDigital Cinema to theaters in the coming years,the use of release prints overall is likely to declinein favor of Digital Cinema Packages (DCP) fordigital distribution to theatres via hard disk, fiberor satellite.

Based on interviews, it appears that within the major studios there is no clear strategy yet fordealing with these new digital assets. Born-digitalassets are being generated in growing amounts.Without a clear plan or direction, managers atproduction companies, post houses and the stu-dios themselves generally seem to be taking thesafest short-term approach, which is to continuethe conventional practice of saving everything forpossible future use, and keeping the assets in theiroriginal format – that is, putting digital cameraoriginals on HDCAM SR tapes, magnetic harddrives and LTO data tapes on a shelf in a cool, dry place until further notice. Some studios are

TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L3.6 The Impact of Digital Technology on

Motion Picture Archiving continued


recording “digital master” files of the completed motionpicture to LTO data tape cartridges and putting themon the shelf next to the HDCAM SR videotape cassettes.It is a reasonable and prudent interim tactic, but not along-term strategy.

New types of content are not suited to film preservationEven studio executives who firmly believe in the wisdomof film-based “store and ignore” archives realize theymust eventually embrace digital archiving and reducetheir exclusive dependency on “film in the freezer” forlong-term preservation of corporate media assets. Theirown marketing and sales teams, tracking new demandtrends and innovating to generate new revenue opportu-nities for their business units, are driving changes in theformats and mix of the commercial media products manufactured by the studios. Full-length motion pictures for theatrical release will continue, of course, butthe non-theatrical release versions sold by studios accountfor much larger percentages of their global mediabusiness [Galloway]. Some of the non-theatrical releaseproducts will still be derivative of the cinema original, butmany will not. This will affect the choice of elementsthat studios need to put in their libraries and archives andhow they will be used in the future.

One studio executive explained that he expectschange in archival rationales will largely be driven bythe changing form factor of content. Eighty percent of this studio’s business now is about feature-length cinema, 90 to 135 minutes per title, and television-show length: 22 minutes and 44 minutes per episode.Naturally, these are the primary “content” beingarchived today by studios. But a growing volume ofmaterial created at the desktop level is neither for fea-ture movies nor television programs. There is growingdemand for short videos and animation. New digitalformats for Internet distribution, elements producedoriginally for the World Wide Web, and content targeting small portable media players are not yet being consistently archived. As customers get more of theirentertainment from more types of digital channels, themedia elements that are created by the studio will besmaller, more varied, and more numerous. A decreasingpercentage will be film-based or even suitable for filmrecording. An increasing percentage of a studio’s productive output of commercial media assets will be“born digital” and cannot be preserved through tradi-tional film archiving practices. This is presenting thestudios with new archiving questions.

The theatrical release of a new movie is increasinglyaccompanied by the simultaneous release of companionvideo games by the games division of the studio inorder to get greater customer awareness and higher saleswithin the demographics targeted by both theatricalmarketing and games marketing. This raises new ques-tions about how the studio should archive the digitalassets created for the game such as the digital characters, computer models, scenery and software programs thatdetermine the game’s interactivity and “play value.”Preservation on film is not even an option in this case.

Long-term viability of film as a preservationmedium is also at risk because of overall filmmarket trendsToday’s large-scale “day-and-date” release patterns haveactually increased the use of intermediate and print filmstocks. However, the accelerating conversion of cinema todigital distribution following the recommendations ofHollywood’s Digital Cinema Initiatives (DCI) consortiumwill erode the market for film release prints and interme-diate film. In parallel, the emergence of Hollywood-gradedigital cinema cameras will likely cut into sales of cameranegative film. All these market trends will put downwardpressure on sales volumes for film manufacturers, film laboratories and suppliers of film-processing chemicals. As demand shrinks for any consumable technology, manufacturing loses economy of scale, product availabilitydecreases, prices increase and quality control suffers, further depressing customer demand.

The manufacturing base for high-quality 35mmentertainment film is already shrinking and has consoli-dated to just three remaining vendors, Kodak, Fujifilmand Agfa, although Agfa produces only print film andspecialty black-and-white stock for sound applications.All are big companies with proud histories as technicalinnovators and market leaders. But none are likely tomake significant new investments in film R&D or manufacturing improvements, given the sinkingdemand for their film products. While Kodak, Fujifilmand Agfa continue to supply high-quality, reliable filmstocks and film chemicals, their managements will notcommit their firms to the entertainment film business“always and forever.” Nor would their shareholders welcome such a commitment to a “sunset” market.

The consumer film market is also collapsing due tothe enormous popularity of digital cameras. Accordingto IDC market research, worldwide manufacture ofconsumer film peaked at 80 to 90 million prints a yearin the late 1990s, but had declined to about 40 million


TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L


by 2005, and is continuing to drop 20 to 30% peryear [Hogan]. This weakens yet another pillar ofthe film business that has historically offered valu-able economy-of-scale advantages to the largestfilm manufacturers.

Several people interviewed for this reportacknowledged that the demise of film is a long-term eventuality, but do not expect film to disappear in the next decade. However, studios planning their long-term archiving strategies must recognize the risk that analog film archiving ofnew titles may become more expensive and/orstop being a viable option altogether in the future.Is it prudent to build long-term preservation infrastructure based on a medium that, if nottotally obsolete, may only survive in niche markets –like motion picture archiving? When YCM separations reach the end of their archival life,archivists entrusted with these valuable corporateassets must consider whether it will be better to

migrate them to another generation of film stock,or to migrate them to a future digital format withits to-be-determined preservation methodology.

Some old film assets will be selectivelyadded to digital archivesThe wholesale migration of major film archives todigital storage is such a large and expensive under-taking that no studio appears to be consideringthis currently, at least for archival preservation inthe strict sense. It seems likely that the studioswill start to consider digitally scanning some oldercontent to protect irreplaceable film elementswhen they become dangerously fragile or deterio-rated. Some old film assets will also be digitizedfor commercial exploitation on a deal-by-dealbasis, because once converted from analog film todigital files the content can be more easily manip-ulated and re-purposed to generate new revenues.

TH

E

TR

AN

SI

TI

ON

T

O

DI

GI

TA

L3.7 Television


ALTHOUGH THIS REPORT IS AN INVESTIGA-tion of digital archiving and access issues from the motionpicture industry’s perspective, there is no denying the inti-mate connection between production and consumptionof theatrical motion pictures and television programming.In fact, every major Hollywood studio also has significanttelevision activities, and it is worthwhile to look at whathas happened and is happening in that area.

History of Television ArchivingAccording to a 1997 report by the Library of Congress[US, LC, NFPB, Television/Video], historically, fewtelevision programs held by the major studios and networks have been destroyed due to deliberate deci-sions or policies. In fact, the growth of non-broadcast distribution channels, consumer packaged media sales,and overseas markets for American TV programs, hasencouraged systematic preservation. All the major studios, even in 1997, had implemented asset-preserva-tion programs for their prime-time programming thatincluded both film and videotape assets. The reason forpreserving these programs was explicit: they representreal assets of value to their corporate owners.

Network news divisions, even in 1997, were havingdifficulty preserving all their programs because of thesheer volume they produced every day. They focusedon preserving what they judged was valuable for thedaily production needs of their reporters and editors,more than on keeping historically complete archives ofall the news they broadcast.

The oldest television archives are on 16mm and35mm film. 16mm film was phased out after ElectronicNews Gathering (ENG) cameras and compact video-tape recorders were introduced in the 1980s. 35mmfilm is being replaced by HDTV acquisition even for premium programs. Many local stations simply discarded their 16mm film libraries when they convertedto U-matic videotape, leading to gaps in the publicarchives of local news between 1950 and 1975. Eventoday, most local news content is not saved more than afew weeks before the videotape is recycled. But thelargest, most progressive broadcasters have been migratingtheir archives from film to videotape, and from video-tape to all-digital archives using general-purpose

Information Technology (IT) infrastructure for the past 5 to 10 years, still a “work in progress” accordingto many in the field.

Since shifting from 16mm film acquisition tovideotape recording (which is recognized to be a muchless durable medium than film), the broadcast industryhas repeatedly chosen to adopt tape formats to takeadvantage of technical advances that offer near-termoperational, economic and/or quality improvements.

Videotape recorder vendors have engineered manyimprovements since AMPEX introduced the first commercial videotape recorder in 1956, the VRX-1000with its proprietary 2'' Quadruplex tape format. Sincethen, there have been more than 60 different videotapeformats. Scanning methods, signal encoding and imageformats have evolved rapidly. Image and sound recordingquality has gone up steadily while size and cost havecome down just as steadily.

Over the years, competing vendors have fought“format wars” for market share, churning out new andbetter devices that users have sequentially adopted totheir own advantage. This has left video archives full of many incompatible formats that run only on obsolete devices, requiring migration from old tape formats to new formats to access the value of the assetsin the archives.

Modern digital videotape, such as HDCAM SR, areclaimed to have a shelf life, under recommended environ-mental conditions, of up to 30 years according to manu-facturers’ commercial literature. The HDCAM SR formatis still far too young to confirm this longevity empirically,and there is no assurance that functioning HDCAM SRtape drives will still be available 30 years in the future.

Today, professional videotape manufacture is limitedto a few very large companies that have, in the past, beenable to effectively leverage the consumer market forvideotape to achieve economies of scale in manufacturingand R&D. But the consumer market for videotape has declined greatly in the past decade as Hollywood-packaged media shifted from VHS to DVD. When this consumer trend is combined with the accelerating conversion of broadcast infrastructure and workflow totapeless, file-based operations [Kienzle, “Breaking,” 1], itis likely that videotape as a recording medium will itselfbecome obsolete in the not-too-distant future.

Since shifting from 16mm film acquisition to videotape recording, the broadcast industry has repeatedly chosen to adopt tape formats to take advantage of technical advances that offer near-term operational, economic and/or quality improvements.


CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S

4 Current Practice • Other Industries

ONE OF THE MOST OFTEN-ASKED QUESTIONS WHEN DISCUSSINGHollywood’s digital archival issues is, “What are other industries doing?” Many industries have already adopted strategies for the preservation of their digital data collections, and while Hollywood’s digital assets are large in number and size, they arenot unique in these respects. Potentially, Hollywood does not need to invent digitalarchiving from scratch, so the risks of trying new approaches can be kept relatively lowby studying others.

There are several areas of modern society that have large collections of mediaassets of various types with archival requirements similar to the needs of the cinemaindustry. While the digital motion picture assets that the Hollywood studios want toprotect are exceptionally large on a per-title basis, the scale of Hollywood’s archivingrequirements is not so different from that of institutions in other domains which alsorequire long-term preservation of very large volumes of valuable pictures, sounds, textand other types of data in support of their missions.

Advanced visualization technologies have always been supported by three “pillar”industries that drive the state of the art: entertainment (cinema and publishing),defense/intelligence, and science/medicine/education. Historically, they have all usedanalog imaging techniques developed for their particular needs, with little dialog ortechnical cross-fertilization between them. However, with the widespread adoption ofhigh-quality digital imaging, all three pillars are moving away from film to common digital platforms applied to their different purposes. All three have a growing need fordigital archiving of still and moving images. All are facing similar challenges in termsof infrastructure, workflows and requirements for long-term preservation. Thedefense/intelligence and science/medicine/education communities are already operatinglarge digital image archives and can provide valuable reference for Hollywood studiosinitiating their own digital archiving programs. All of these large-scale, long-term digital archives have refined their system designs over the years to accommodate dataingest, search and retrieval, as well as relatively efficient and reliable data migration,file format updating, auditing, quality control and (when used) discard/transferprocesses needed to insure the accessibility and integrity of their digital assets. Theyalso anticipate long time horizons: 50 to 100 years, or even “permanently” in somecases; that is, for the life of their particular enterprise, assuming adequate funding.

Other media archives are already transitioning from analog to digital. Many largepublic libraries and archives with extensive collections comprising many media typesare preserving their most recently acquired content digitally because more and moremodern media are originally produced digitally, “born digital” assets that are deliveredto the library on some kind of digital storage media or directly as data files via a computer network. Users have generally appreciated the faster, easier accessibility ofthe digital collections offered by the libraries. In response, librarians and archivists are digitizing their most important (most popular) analog assets to make them more accessible, too. Other assets are being converted from analog to digital when the analog media are deteriorating, putting survival of the content at risk in the absence of a reliable digital preservation strategy. Similar trends can be seen in several commercial media industries as well.

The scale ofHollywood’sarchivingrequirementsis not so different from that of institutions in otherdomainswhich alsorequire long-termpreservationof very largevolumes ofvaluable pictures,sounds, textand othertypes of datain support of their missions.

CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S4.1 Corporate America


4.1.1 Sarbanes-Oxley Act Requirements

IN TERMS OF THE TECHNICAL DIFFICULTIESinvolved in long-term preservation of digital assets, themost significant difference between corporate recordarchiving and Hollywood archiving is the intendedduration of archival preservation. For example, theSarbanes-Oxley Act of 2002 (SOX), passed as a result ofthe corporate accounting scandals at the turn of thiscentury, only requires preservation of certain types ofcorporate data for seven years, a term marginally withinthe life cycle of a single generation of digital storagetechnology. SOX affects mostly transactional data, whichmeans that the archive period starts at the time of trans-action. The archived data is always rolling over as newdata replaces old data that can be discarded after theseven-year mandatory archival period. There is a statu-tory requirement for “protecting the unalterability” ofthe archives, so implementation emphasizes the use of“Write Once, Read Many” (WORM) storage, audittrails, rigorous access control, data authentication tech-niques and legal compliance.

This contrasts to the Hollywood goal that digitalmotion picture archives be preserved for 50 to 100years, comparable to existing film archives. That is alonger period of time than any digital technology available today can reasonably support without usingspecialized digital preservation strategies such as datamigration, discussed later in this report. Nonetheless,properly designed corporate data archival systems imple-ment the defensive data preservation strategies describedin Requirements for Digital Preservation Systems and theNational Research Council’s Recommendations for a Long-Term Strategy [Rosenthal 3; Natl. Research 59-69].

A further contrasting parameter is the storage volume of corporate data that requires this level ofpreservation. Many of the studies on corporate ITpractices reviewed for this report were sized in the gigabyte and terabyte range, which is substantially lessdata than is generated by a single digital motion pictureproduction. This has significant impact on overallsystem costs, from both initial capital investment andoperating perspectives. This topic is covered in detail inSection 6.

Finally, the tight integration of business systemsand IT infrastructure, as well as the relatively commondata storage requirements across corporate America,enables significant economies of scale and close collabo-rations with IT vendors that are not easily achievable inthe motion picture industry, given the specializednature of motion picture production.

4.1.2 Oil ExplorationIN INTERVIEWS WITH A LARGE OIL COMPANY’Sdata system manager and a data storage service company,it was learned that the oil and motion picture industrieshave something in common: the derived data is morevaluable than the raw captured data. That is, capturedgeological data must be heavily processed before it hasany immediate value – that value being the location andsize of oil deposits to be extracted. The processing algo-rithms improve over time, and that is the incentive topreserve the original captured data: new oil deposits canbe identified using “old” data.

The oil industry has another characteristic in com-mon with the motion picture industry: a typical rawgeological data set can be 200 terabytes (the size ofabout 25 uncompressed 4K digital motion picture masters), and a typical survey of the Gulf of Mexico cangenerate hundreds of data sets, and is normally storedon hundreds of magnetic data tapes.

According to the people interviewed, raw geologicaldata began to be archived more than a decade ago, andthey are experiencing problems that sound familiar: aheavy and undesirable reliance on vendor-specific solutions which limit future freedom of choice, a lack of standard file formats (which enforces single-vendorreliance), ad hoc archiving procedures, no experiencewith data migration, and a need to maintain workingversions of old hardware and software to guaranteeaccess to valuable data. Therefore, standardized archival practices and data formats are a long-term goal for this industry.

The most significant difference between corporate record archiving and Hollywood archiving is the intended duration of archival preservation.


CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S

4.2 Government and Public Archives in the U.S.

THE LIBRARY OF CONGRESS’ NATIONAL DIGITAL INFORMATIONInfrastructure and Preservation Program (NDIIPP) initiative and the National Archiveand Records Administration’s Electronic Records Archive (ERA) program have bothput emphasis on collaborative problem solving, drawing on the opinions of expertsfrom many fields and providing forums for valuable exchange of information thatserves the purposes of these institutions and contributes to general understanding ofthe challenges and possible solutions to large-scale institutional preservation of digitalassets. The Academy participates in both of these federal government initiatives, as amember of NARA’s Advisory Committee on the Electronic Records Archive (ACERA)and a partner in the Library’s Preserving Creative America program under NDIIPP.

4.2.1 Library of CongressACCORDING TO ONE VETERAN AT A LARGE STORAGE MEDIAmanufacturer, in the late 1980s the U.S. Library of Congress said it wanted a 200-year archival life for its digital assets. The vendor’s engineers went to work onaccelerated aging tests to try to meet the Library’s goals. However, after several years it became clear that no digital storage scheme available then (or now, or in the foreseeable future) can be sustained for 200 years. The Library realized that digitalmedia are so ephemeral and digital technology changes so rapidly that long-term digital preservation was going to need a new approach.

In December 2000, Congress appropriated $100 million for the NDIIPP collabo-rative project in recognition of the importance of preserving digital content for futuregenerations. Led by the Library of Congress, NDIIPP has generated digital archivingguidelines that are useful for any organization that is formulating its own strategy tocollect, archive and preserve growing amounts of digital content for current and futuregenerations, especially materials that are created only in digital formats. NDIIPP setfive initial goals for the Library of Congress and, by extension, for any organizationfaced with digital archiving challenges [US, LC, Digital Preservation]:

1. Identify and collect at-risk “born digital” content that is created only in digital form, before it is lost, misplaced, goes obsolete or becomes corrupted.

2. Build and support a network of partners working together to preserve digital content. The task of saving digital assets is too large for solitary efforts.

3. Develop and use technical tools and services for digital archiving.

4. Encourage development of strategic policies to support efficient and reliable preservation of digital information. Document the rules, and educate the staff; technology is only part of the problem.

5. Show why digital preservation is important for everyone in the enterprise. Savinginformation, especially the right information, has to become everyone’s task.

The Library of Congress has an additional motivating factor for the developmentof digital preservation technologies and practices: its National Audio-VisualConservation Center (NAVCC), located in Culpeper, Virginia, will house the entirecollection of the Library’s Motion Picture, Broadcast and Recorded Sound Division.The collection contains increasing amounts of digital materials, and the NAVCC’s digital storage system is expected to ingest over 8 petabytes (equivalent to about 1,040uncompressed 4K digital motion picture masters) per year when fully operational [US, LC, Natl. Audio 15]. Furthermore, as the repository for mandatory copyrightdeposits, the NAVCC system must consider the copyright term of 120 years or longer.

It is clear that the digital preservation concerns of the Library of Congress arequite similar to those of the Hollywood studios.

The task of saving digital assets is too large for solitaryefforts.

CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S4.2 Government and Public Archives in the U.S. continued


4.2.2 National Archives and RecordsAdministration

THE U.S. NATIONAL ARCHIVES AND RECORDS Administration (NARA) is responsible for preserving allofficial government records, both to protect the recordsas official history and to make them available for futurereference. NARA operates both classified (secret) andunclassified (open) archives. By NARA’s estimate, only 1 to 3% of the documents generated by the federal govern-ment are significant enough to be added to the archives.

NARA’s current digital holdings are diverse. A fewdata files were originally created as early as World War IIand reflect punch card technology in use since the 1800s.An even smaller number contain information from the19th century that has been converted to an electronicformat. However, most of the electronic records inNARA’s holdings have been created since the 1960s.

As the 21st century began, NARA planners realizedthat going forward more and more official governmentbusiness will use electronic records that NARA itselfwill have to accept, catalog, search, give access to andpreserve “permanently” in a new kind of digital archivecapable of handling thousands of formats and trillionsof data objects. NARA recognized that these are verycomplex problems requiring long-term planning, andtherefore established the Electronic Records Archive(ERA) program to meet their visionary goals: preserveany type of record, created using any type of applica-tion, on any computing platform, from any entity inthe federal government or any donor; and provide dis-covery and delivery to anyone with an interest and legalright of access, now and for the life of the Republic.

NARA archiving responsibilities are mandated bystatutory requirements to preserve all official govern-ment records, with rules that compel record creators todeliver assets to the Archivist of the United States within certain time limits. The ERA team at NARArealized they could not do or think of everything them-selves, so ERA established a network of partnershipswith computer scientists, engineers, information management specialists, archivists, industry experts and professionals. Through workshops, symposia andfunded research projects, the ERA’s strategy has been toattack the critical preservation problems as the first priority, defining the requirements in terms of the “lifecycle” management of records. They want to usecommercially viable, mainstream technologies beingdeveloped to support e-commerce, e-government andthe next-generation national information infrastructure,aligning NARA with the overall direction of IT in theU.S. government, and in the process perhaps leadingthe U.S. government’s IT practices to align better with NARA’s essential archiving mission. As such,

NARA/ERA is in a position to dig deeply into theissues of long-term digital archiving and help build consensus in the field.

The ERA project is phased, with the initial goal ofaccepting the electronic records of President George W.Bush’s administration when he leaves office in January2009. NARA expects to process and ingest over 800million email messages and attachments into the ERAsystem at that time, and is currently working with four other federal agencies to develop and test the basesystem. Although the system is still in development,NARA believes that the use of standardized file formatsand metadata, as well as automated ingest and metadataharvesting, are critical to the system’s long-term success.

4.2.3 Department of DefenseTHE ACADEMY’S PARTICIPATION ON THENational Archives’ ACERA committee provides a view into the digital data management activities of other government agencies, including the Department of Defense(DoD). One of the most relevant presentations from themotion picture industry’s perspective was an overview ofthe DoD’s Advanced Distributed Learning Project (ADL),which is a system designed to make all of the department’saudiovisual training materials accessible and reusable acrossthe entire department. The volume of training materials islarge, as is the variety of media types, so the ADL systemhas had to address a number of challenging issues rangingfrom basic issues of digital storage to more complextopics such as metadata and digital object registries.

While one might consider the ADL system more ofa digital library than a digital archive, there is a tremen-dous overlap between technologies and practices used inthis system and those the motion picture industry willneed to implement in the future.

Also interviewed were representatives from theOffice of the Director of National Intelligence, whoserves as head of the Intelligence Community. Theyexpressed much interest, both on the part of this officeand the DoD, in collaborating with large producers andconsumers of digital media to develop standardized fileformats, especially with respect to metadata. Theystrongly believe that it is not economically feasible forsingle organizations, even as large as the DoD and thosethat are part of the Intelligence Community, to developthese technologies on their own. As it is, for varioushistorical, organizational and technical reasons, theDoD and intelligence communities are populated withmany large digital-imagery archives that have emergedwithout comprehensive planning for long-term preserva-tion and suffer from interoperability barriers betweenarchives and agencies. Their stated recommendationwas, “Don’t let this happen in Hollywood.”


CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S

4.3 Medical

“ “

“It is exceedingly rare that fundamentally new approaches to researchand education arise. Information technology has ushered in such a fundamental change. Digital data collections are at the heart of thischange. They enable analysis at unprecedented levels of accuracy andsophistication and provide novel insights through innovative informationintegration. Through their very size and complexity, such digital collections provide new phenomena for study. At the same time, suchcollections are a powerful force for inclusion, removing barriers to participation at all ages and levels of education.”

THE HEALTH INSURANCE PORTABILITY AND ACCOUNTABILITY ACT of 1996 (HIPAA) requires hospitals, clinics, medical equipment rental companies,physicians’ networks, dentists, drug stores, medical insurance companies, medicalbilling companies and nursing homes to preserve and protect the privacy of electronicmedical records, including diagnostic medical images. HIPAA requirements are, inpractice, designed more to protect privacy and promote operational efficiency thancomprehensive archiving. The motivation for archiving medical data comes from itsbenefits for research and education. According to a report by the National ScienceFoundation’s Science Board:

And according to the Mayo Clinic in Rochester, Minnesota, “cine [filming] forradiography has served the medical industry’s needs well since the early 1950s. Forthe first time, it allowed recording of motion studies of the cardiac structures on film.The cine technique has been standardized over the years, both the camera and thedisplay. Cine filming techniques, however, have not advanced, except for new filmproducts with faster emulsions and better-quality films. Video imaging for cardiologyhas made rapid advancements.…With the advent of interventional procedures in thecardiac catheterization laboratory, the need to assess images immediately cannot befulfilled by cine filming because of the requirement for the processing of the filmwith its inherent delays” [Holmes, Wondrow, and Gray 1].

According to the Cleveland Clinic, one of the largest hospitals in the U.S. thatmaintains a substantial digital image archive supporting both its clinical and researchactivities, much of the medical field began its conversion from film imaging to digitalimaging a few years after the Holmes paper was published in 1990. Currently, all ofthe Cleveland Clinic’s medical imaging (including radiology) is done digitally, and its plan is to keep all digital images forever for historical trend analysis and research.The current film holdings are not digitized because it is not cost-effective, and the old films are stored in a cool and dry warehouse.

The Cleveland Clinic’s archive currently stores 1 petabyte of digital data, com-posed of objects such as chest images that are about 20 megabytes per image, andmotion clips that are 500 gigabytes per patient. Image data compression is not usedbecause life-or-death medical decisions are made based on this data. The archive isgrowing at a rate of 3 terabytes per week, which will double its size in the next year.Image pixel counts have increased and more images per patient are being made, sothis growth trend is expected to continue.

Their storage strategy is to store the most recent data on an array of magnetichard drives, and there is currently 100 terabytes of such online storage. The biggestproblem with this system is that the disk lifecycle is only three years – every diskdrive must be replaced after that interval. Hardware is not the only cost; all of the

Digital medicalimages are“bornarchival”...all of apatient’s identifyingdata is collectedbefore thediagnosticimage is captured, and the metadata isforever associatedwith a capturedimage.

CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S4.3 Medical continued


data must be copied when the hardware is replaced, andas the archive grows, the time required for copying isgetting longer and longer. Once the data ages to a certain point, it is automatically transferred to a datatape library, with every tape automatically evaluatedevery 90 days. The reliability requirement is zero errorsper 100,000 operations. A secondary archive is located12 blocks away from the primary archive at the Clinic,and the two are linked by a fiber-optic connection.

The medical industry has some experience with fileformat standardization, but it was not particularly positive. When digital medical imaging devices wereintroduced in the 1970s, all vendors had proprietary file formats that were designed to lock their users intotheir technology, and it was a successful strategy for the vendors. When the users wanted to move databetween vendors, the American College of Radiologyand the National Electrical Manufacturers Association collaborated on the development of the Digital Imaging

and Communications in Medicine image format stan-dard, or DICOM [Digital Imaging 5]. The problemwith DICOM, reportedly, is that the standard is implemented differently by each manufacturer, and proprietary extensions have been added by manufacturers,so that interoperability is still not achieved. For thisand other reasons, archiving of medical images is stilldone in proprietary image formats.

It is interesting to note that digital medical imagesare “born archival” with respect to essential metadata.That is, all of a patient’s identifying data – name,address, date of birth, attending physician, billingaccount, etc. – is collected before the diagnostic imageis captured, and the metadata is forever associated with a captured image. This contrasts with the motionpicture industry’s practice of generating metadata after image capture, and the associated downstream difficulties with metadata management.


CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S

4.4 Earth Science

EROS storesfilm today in archives for the samereasonHollywoodsaves film:film can bereasonablypreserved for100 years ormore.

THE CENTER FOR EARTH RESOURCES OBSERVATION AND SCIENCE(EROS) is a data management, systems development, and research field center for theU.S. Geological Survey’s (USGS) Geography Discipline. Organizationally, the USGS is a bureau of the U.S. Department of the Interior. The archive contains aerial photography and satellite remote sensing data of the Earth’s land surface. The EROSmission is to preserve this data “permanently” and make it easily accessible and readilyavailable for study. Residing in the USGS’ EROS Data Center near Sioux Falls, SouthDakota – one of the largest computer complexes within the Interior Department – is theNational Satellite Land Remote Sensing Data Archive (NSLRSDA), a comprehensive,permanent, and impartial record of the planet’s land surface derived from 40+ years ofsatellite remote sensing. Aside from the larger question of change at the global scale,NSLRSDA permits scientists to study water, energy, and mineral resource problems over time; to help protect environmental quality; and to contribute to prudent, orderlymanagement and development of our nation’s natural resources.

Over the past three decades the U.S. government has invested money to acquireand distribute data worldwide from the Landsat series of satellites – more than 630 terabytes of which are held at the EROS Data Center. The archive also includes more than 28 terabytes of data from the Advanced Very High Resolution Radiometer(AVHRR) carried aboard National Oceanic Atmospheric Administration's polar orbiting weather satellites, and more than 880,000 declassified intelligence satellitephotographs.

The primary objective of NSLRSDA is to preserve entrusted data records “permanently” and to distribute this data on demand to a worldwide community ofscientific users. As a result, the EROS Data Center has become a world leader notonly in techniques of archiving remotely sensed data, but also in getting the data toend-users quickly, in forms they can use, at costs they can bear. According to thearchivist at EROS, every advance in online distribution, in storage media, in applica-tions research, or in cost-saving delivery technologies means more people can use thedata. As demand increases, user expectations about delivery times and efficiency rise.

EROS archives also contain approximately 4 million satellite images of globalscale and 8 million aerial images of the U.S. Images held are stored both on film anddigital media, but almost all new images are digital. In 2004, the archive included80,000 pieces of film, and by early 2007, the film archive had grown to 110,000 piecesas other agencies send their collections to EROS for archival preservation and scientificaccess. Film assets are preserved in climate-controlled film vaults that have been inspectedby the National Archives, which estimated a 100+ year shelf life for these film assets.

In 2004, EROS archives held roughly 2 petabytes of digital image data “nearlineand online” in robotic data tape library systems and magnetic hard drive arrays. It took30 years for the archive to reach this size. In 2004, the archive was growing at the rateof 2 terabytes per day, and EROS forecast a doubling of the digital archive in just fouryears. As of this writing, EROS archives hold more than 3 petabytes, on track to matchthe 2004 forecast, and the archives continue to grow at the rate of 2 terabytes per day.

EROS stores film today in archives for the same reason Hollywood saves film:film can be reasonably preserved for 100 years or more. The question is: at the end of 100 years’ life of film in archives, does one make a new film copy or make a digitalcopy? Currently, EROS does not keep high-resolution scanned data from originalfilm images; it scans on demand from the film archives, with full-resolution scans atapproximately 7904 x 8512 pixels, approximately 800 megabytes per frame, at a costof $20 to $30 per frame.

EROS provided some interesting observations from its experience building andoperating a large digital archive: the operating cost is proportionate to the number oftimes the data is read, and the risk of losing data is proportionate to the number of

CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S4.4 Earth Science continued


times data tape is accessed. EROS expressed similaropinions to other digital archive operators that opticaldisks appear attractive from a cost perspective but theydo not have suitable long-term reliability characteristics,and power consumption of magnetic hard drives is agrowing cost. Furthermore, EROS said it was impor-tant not to become dependent on a single technology.They related a story about two different, large digitalarchives that chose a technology called CREO whichhad but one supplier. The supplier went out of busi-ness, requiring an immediate and unplanned migrationfor both of these organizations – one European and oneCanadian – at a cost of millions of dollars. Theyemphasized that this was a lesson not to be forgotten.

EROS also has experience with data migration.They prefer to invest in the migration costs of a collection rather than potentially experience the “cost”of losing a collection, and they believe migration worksas a possible strategy to ensure long-term access. Theirearly data migrations were very expensive, took a longtime to execute and were performed at 7- to 10-yearintervals before 1992. Since 1992, migration has takenplace every 3 to 5 years, and based on lessons learnedand investments in robotic library systems (they aremoving toward the SUN T10000 tape technology) and

other efficiency-enhancing technologies, migration atEROS is getting easier, faster and less expensive, evenafter including 100% read-after-write data verificationduring migrations.

Since the events of September 11, 2001, a higherpriority has been placed on a full offsite system – athird archive – for disaster protection. But EROS alsolearned as a result of the 9/11 attacks that air travel canbe disrupted for extended periods. Therefore, it is moredesirable to build offsite data storage within driving dis-tance. Today, EROS has only a few terabytes offsite,but is planning to establish a 100% redundant offsitearchive, such that EROS will have three archives:

• First copy – near-line/online, on a robotic tapelibrary or on magnetic hard drives

• Second copy – offline “basement tapes”

• Third copy – physically offsite

Their goal is to keep the best-quality data, and fewestversions of the data in the archives.

In the opinion of the people interviewed at EROS,it is inevitable that distribution library and long-termpreservation functions/systems will merge.


CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S

4.5 Supercomputing

THE SAN DIEGO SUPERCOMPUTER CENTER(SDSC) operates three supercomputers for the nationalresearch community and provides supercomputing services for a range of extreme computing needs such as astrophysics visualization, bioinformatics and other science and engineering disciplines. The SDSC also operates a large hybrid storage system with 2.5petabytes of online magnetic disk storage, and 5petabytes of near-line data tape storage, with a capacityof 25 petabytes for the robotic tape storage system. Forperspective, 25 petabytes is enough to store over 3,000uncompressed 4K digital motion picture masters, orapproximately 5 to 6 years of MPAA-rated motion pictures. SDSC’s storage volume is doubling every 14months, and they expect it to grow to 10 petabytes bythe end of 2008.

According to those interviewed at the SDSC, data migration, or the copying of data to new storagedevices and media from those becoming obsolete, is afact of life for them, “like painting the Golden GateBridge,” in that one must continually repeat the processto avoid decay. SDSC’s migration period is five years,and they say one of the benefits of migration is that thenewer storage technology is faster and denser than theold technology being replaced.

One of the factors determining the migration period is concern about “bit error rate” (BER), which isa fundamental measure of a storage medium’s reliabilityand integrity of the recorded data. BER generallyincreases with the age of the storage media, and there-fore the oldest data tapes in SDSC’s collection are only6 to 7 years old. However, they say the BER is nearlyirrelevant to long-term preservation and that the pri-mary causes of digital archive loss are human error andmagnetic disk hardware failures. SDSC relies on data

migration and a program of ongoing data verificationto ensure data preservation. But there is a level ofuncertainty to data verification because the act of reading data for verification increases the probability oferror, but not reading data for verification can allowlatent errors to accumulate unnoticed. This topic wascovered in detail at the 2006 Eurosys Conference[Baker, et al. 3].

SDSC’s biggest concern is not with technologicalobsolescence – they think the biggest risk to datapreservation is gaps in funding for system maintenanceand data migration. To address this concern, SDSCexpects to begin allocating storage costs to system users, instead of just billing for computing power as iscurrently the practice.

Computer networking is a factor in the SDSCarchiving picture. In a collaboration among SDSC, the National Center for Atmospheric Research locatedin Boulder, Colorado, and the University of Maryland,these three organizations have agreed to share archiveresources that will only be accessed if one party loses its primary copy. Despite some opinions thatarchives should not connect to networks, those inter-viewed at SDSC recommend that operators of digitalarchives learn how to use networking because they feelit is faster and more efficient for debugging and problem solving.

SDSC is also collaborating with the Library ofCongress on a pilot “third party repository” projectdesigned to address certain risk scenarios, e.g., stream-lining the number of file formats and file systems tosupport. In general, they think very-long-term datapreservation and “cold storage” digital archiving areunsolved challenges suitable for future research.

The primary causes of digital archive loss are human error and magnetic disk hardware failures.

CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S4.6 Summary


THIS SURVEY OF OTHER INDUSTRIES WITH LARGE DATA STORAGEand long-term preservation needs revealed a set of common issues that also arise inthe motion picture industry’s conversations about digital archiving, and it is worth-while to summarize these issues as well as the advice offered by those who by nowhave a substantial amount of firsthand experience.

There is both consensus and disagreement among those interviewed on key issuesof long-term digital data preservation.

4.6.1 Consensus ViewThere is general agreement on the part of those interviewed from outside the motionpicture industry that:

• Multiple copies of important digital data should be maintained

• The stakeholders, not the vendors, should drive requirements and standards

• The total cost of ownership is much more than just media costs

• The cost of labor, and secondarily the cost of electricity, not technology, are the limiting economic factors in digital archives

• There is definite economy of scale in digital archiving systems

• The number of file formats and file systems used should be minimized (and they should be chosen carefully) to keep labor costs down

• An extensible file system should be chosen to keep down long-term management costs

• Economics force an ongoing assessment of future value of assets each time a major data migration is done

• Good project management is essential in all migrations

• Unproven or exotic technology should not be used

• Biodiversity, or spreading technological risk across several different technologies, is important; i.e., do not be dependent on a single vendor

• It is very difficult to change vendors after even one year of using a new system, so the initial choice is critical. Make sure to negotiate up front in order to continue using archival system software after its license expires

• Currently there is no digital alternative to analog film archiving if the goal is “store and ignore” long-term preservation for 50 to 100 years

Of particular note is that everyone interviewed for this report agreed that no onecan make a perfect choice of what to save and what to discard, or how valuable an assetwill be in the future. This issue, as it applies to the motion picture industry, is dis-cussed in much more detail in Section 6.3.

“Don't makethe same mistakes we made by letting all the different vendors create proprietaryformats.”


CU

RR

EN

TP

RA

CT

IC

E

•

OT

HE

RI

ND

US

TR

IE

S

4.6 Summary continued

4.6.2 Unresolved IssuesIn discussions with people outside the motion picture industry, a wide range of opinionssurfaced on a number of issues:

• Is data migration most effectively done in-house or on an outsource basis?

• Is data compression an important technical consideration? (Although those interviewed all supported “no compression” unless an asset is “born compressed”)

• What data should be saved, and what should not be saved?

• What level of geographical separation should be achieved; i.e., how far apart is far enough?

• Should a primary and backup archive system be connected, or not be connected, by a network?

• Are standardized file formats necessary?

• What is the best digital preservation strategy?

The issues of standardization and policy-setting merit further discussion.

Standards and PolicyWith respect to standardizing file formats, some recommend converting everything to a normalized or universal archive file format upon ingest, usually based on a widelyaccepted standard of the day. But there are many contra-indicating examples of standardsnot staying “standard” or going obsolete. Some recommend archiving in the originalfile format submitted to the archive. This avoids having to normalize formats uponingest. This approach requires the archive to be capable of holding many different fileformats, including, potentially, original data creation (acquisition) formats, intermediatedata processing (postproduction) formats and final data delivery (distribution) formats.

Returning to the motion picture industry, one of the major Hollywood studios stated quite clearly that it is not concerned about picking an eternal universal archivingformat, nor is it worried about compatibility between different archives. When theyneed to, they say, they will convert, or “transcode,” formats. They say it is better if converting can be avoided, but it is not an insurmountable obstacle when necessary. It “just costs money and time.” This studio recognizes that it does a lot of formattranscoding today, and they assume this transcoding requirement continues as a “fact of life,” but probably will become even easier and faster in the digital future.

Some of the people interviewed for this report believe the stakeholders – the ownersand the archivists – should try to influence any standardization process to their advan-tage because they will be paying for the long-term preservation costs and thus shouldhave a large say in what is created. They said, “Don’t make the same mistakes we madeby letting all the different vendors create proprietary formats.”

Setting archive policy involves utilizing common records-management practices(Association of Records Managers and Administrators and the Society of AmericanArchivists are good places to start) and utilizing them along with as many others as possible. Having stakeholders involved will increase the chances of gaining consensus.Implementation will involve an ongoing educational element. This is never an easy task,and although it becomes less onerous over time, it never goes away. Gaining acceptanceand support from the highest management levels is essential for success. Without upper-management support, standardizing archival policies will be very difficult.

AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T5 Archiving in the Changing Environment


DIGITAL ARCHIVING IS NOT JUST A MATTER OF ARCHIVING DIGITALassets by putting digital storage media (magnetic hard drive, magnetic data tape or optical disk) on a shelf next to existing analog archives (film). The long-term accessibilityof digital assets on magnetic data tape or magnetic hard drives or optical disks cannotbe reliably protected for the long term just by keeping the humidity and temperature ofthe archiving environment within an acceptable range. Archiving digital data requires a more active management approach, and a more collaborative partnership among producers, archivists and users to exploit its full benefits.

Accessing the data stored on digital media requires access to the digital tools that“go with” the archived data. For example, early digital data from the NASA Vikingprobes launched in 1975 was transmitted from Mars back to the Jet Propulsion Lab inPasadena, California, where it was recorded on magnetic data tape, analyzed by scien-tists at the time and then archived in a cool, dry data warehouse and left undisturbeduntil 1999 when USC neurobiologist Joseph Miller asked NASA to check some of theold Viking data. NASA found the tapes he requested, but could not find any way toread them. It turns out the data, despite being only about 25 years old, was in a formatNASA had long since forgotten about. Or, as Miller puts it, “The programmers whoknew it had all retired or died.” Luckily, Miller was able to cobble together about athird of the data and get some useful results off the Viking tapes, but only because healso found a partial set of reference notes and records printed on paper that had beenput away with the tapes [Kushner 3]. Overall, this incident with NASA’s Viking datawas an important warning bell about the dangers of what some have called “data extinction” and stimulated the development of a data reference model called the OpenArchival Information System (OAIS), designed to protect data assets within the U.S.federal government through systematic data migration.

Interactive media, especially when designed for use on custom-made hardware andsoftware, are exposed to another type of long-term threat for digital archiving. Forexample, the BBC Domesday Project was a pair of interactive videodiscs made by theBBC in London to celebrate the 900th anniversary of the original Domesday Book. Itwas one of the major interactive projects of its time, involving the work of 60 BBCstaff, a budget of 2 million pounds and the volunteer efforts of thousands of Britishschoolchildren and teachers. The modern Domesday contained text, photographs,video, maps, data and a controlling computer program to bind it all together. The finalpackage was published on two custom-designed laser disks with the special controllingsoftware designed for the BBC Micro, a popular microcomputer. This software pro-gram was composed of 70,000 lines of custom code written in BCPL, a forerunner ofthe widely used C programming language. Within 15 years, it was impossible to usethe “digital” Domesday, as compared to the original Domesday Book which was hand-written, probably by a single monk in 1086 and which is still readable (in Latin) if onegoes to the UK’s National Archives, where it has been preserved. However, in 2002, aresearch project by the University of Leeds and the University of Michigan managed tosuccessfully emulate the original BBC system using modern hardware and software, oneof the pioneering efforts in digital “archaeology” that enabled continuing access to old,nearly “extinct” digital media assets.

These ominous examples epitomize the difficulties faced in maintaining digital dataaccessibility over a long period of time. There are several similar stories circulating inthe motion picture industry that ultimately had happy endings, but they foreshadowthe possibility of more dire consequences in the absence of adequate digital preservationpractices. To understand the underlying reasons for these difficulties, it is necessary tounderstand certain technical and operational aspects of digital storage technologies andthe systems built around them.

Archiving digital datarequires amore activemanagementapproach, and a more collaborativepartnershipamong producers,archivists and users toexploit its full benefits.


AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T

5.1 Digital Storage Technology

MANY TECHNICAL TERMS AND ASSUMPTIONSare thrown about in any conversation about archiving andpreserving digital data. The following section presents acondensed summary of practical information on variousstorage technologies, their reliability and other factors thataffect the access lifetime of important digital data.

There are four primary digital storage media in professional use today: magnetic hard drives, digitaldata tape, digital videotape, and recordable optical disk.Solid-state memory devices, such as those used in digital still cameras and more recently in ENG andDigital Cinema acquisition, are not considered in thisdiscussion because their storage densities (and thereforecost-effectiveness) are not likely to make them an influencing factor for motion picture archiving in theforeseeable future.

Magnetic hard drivesAlso called “hard disks,” “hard drives,” or just “drives,”magnetic hard drives have shown an impressive increasein storage capacity over the last 20 years and are the firstchoice for high-speed online storage. The earliest drivesavailable for personal computers stored 5 megabytes (thesize of a single digital photograph from today’s con-sumer digital still cameras) and cost $1,500. As of thiswriting, 750-gigabyte drives are available for $269, andthe annual 30% storage density increase continues.

Long-term magnetic disk storage capacity and costtrends are both favorable from the point of view ofhigh-volume digital data producers. Conventional wisdom is that the cost per bit of magnetic storage isdeclining 40% per year or more. This is a long-term(40-plus years) trend that is expected to continue atleast until 2025 or 2030. In other words, by 2020, iflong trends continue unabated, a terabyte disk can beexpected to cost $7.50 to $15 and a 1-petabyte disk(1,000 terabytes or enough to store over 100 uncom-pressed 4K digital motion picture masters) only $7,500to $15,000. However, after 10 to 15 years, currentmagnetic recording technology could hit fundamentaltechnical barriers, so it is not feasible to estimate harddisk trends beyond this period of time.

It should be noted that magnetic hard drives aredesigned to be “powered on and spinning,” and cannot

just be stored on a shelf for long periods of time. Thedrives’ internal lubrication must be occasionally redis-tributed across the data recording surface through nor-mal operation of the drive, otherwise they can develop“stiction” problems where internal componentsmechanically lock up. New power-saving strategiessuch as Massive Array of Idle Disks (MAID) attempt toaddress this problem at the cost of increased accesstime, although individual drive units still have a limitedoperational lifetime.

Digital Data TapeThe three leading data tape formats for digital archivingare Advanced Intelligent Tape (AIT), Digital LinearTape (DLT), and Linear Tape-Open (LTO). Of thethree, LTO, an open-format tape storage technologydeveloped by Hewlett-Packard (HP), InternationalBusiness Machines (IBM), and Seagate (which spun offits data tape business as Certance in 2000, subsequentlyacquired by Quantum in 2004), is the dominant formatused in the motion picture industry and also has 82%market share in the mid-range tape drive segment[Mellor]. The term “open-format” means that usershave access to multiple sources of storage media products that will be compatible. The high-capacityimplementation of LTO technology is known as theLTO Ultrium format.

LTO Ultrium technology has evolved through several generations. The current LTO4, which becameavailable in 2007, has a native capacity of 800 gigabytesper cartridge (1.6 terabytes using built-in data compres-sion) and a maximum transfer rate of 240 megabytesper second. LTO5 and LTO6, still under develop-ment, are each expected to successively double LTO4’sstorage capacity and data transfer rate. LTO3 and itspredecessor LTO2, each with lower capacities andtransfer rates, are in wide use throughout the motionpicture industry.

Technically, LTO offers faster access times thanDLT. In addition, LTO features larger capacity per cartridge, higher transfer rates, multi-vendor interoper-ability, and a clear multigenerational technologyroadmap that promises two generations of backwardread-compatibility. For example, LTO1 tapes from

Magnetic hard drives are designed to be “powered on and spinning,” and cannot just be stored on a shelf for long periods of time.

AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T5.1 Digital Storage Technology continued


2000 are still readable on LTO3 drives introduced in2004, but will no longer be readable on the new LTO4devices due to practical limits of the physical media and electromechanical drive mechanisms.

Even executives of Quantum, the leading vendor ofDLT tape, agree that LTO tape has won the format warfor large-scale digital archiving. This is why Quantumacquired Certance in 2004 and publicly announced all itsfuture investments in mid-range tape will focus on LTO[Global 10]. As a result, Quantum expects its own DLTtape will have a shorter commercial life than LTO.

According to Sun Microsystems, its Titanium10000 (T10K) data tape system has been developed tomeet the needs of “enterprise-class” storage applications.Enterprise-class storage is better for mission-criticalapplications, such as data centers and valuable archivesbecause the data transfer rate is higher, the cartridges aremore durable, and all components are manufactured totighter specifications for more reliable, consistent opera-tions and longer life cycles. They also point to slightlysuperior bit error rate for the T10K format. On the otherhand, according to some of Sun’s competitors, the enter-prise market segment is being eroded from below by mid-range LTO which has been steadily improving interms of reliability, durability, bit error rate and errordetection/correction to the point that the advantages ofenterprise-class products are less meaningful. They saymid-range LTO is good enough for digital archivingapplications, and significantly less expensive. A 2006study on data tape technologies prepared for the UnitedStates Geological Survey appears to confirm this point ofview [Science Applications 13].

Some industry executives interviewed for thisreport worry that the collapse of the consumer marketfor VHS tape will weaken research and developmentinvestment in magnetic tape in general, and thereforeslow the historical downward dollar-per-bit trendsenjoyed by professional data tape for the past 20 years.They suggest that prices for professional data tape stockand its ingredients such as magnetic coatings, bindersand lubricants will go up as consumer tape volumesdecline. To keep up the pace of progress, data tapemakers may have to invest more in their own fundamental R&D, to be amortized through higherprofessional tape prices.

According to an executive with Imation, a largeprovider of data storage products spun off from 3M in1996, tape manufacturing operations for audio/visualapplications were discontinued by Imation at the time of the spin-off because it had become a low-profit commodity business. Imation saw no danger, however,to its growing and profitable data tape business.

One final note about the large “robotic library” storage systems built around any data tape format: somepostproduction facilities interviewed for this report statethat the benefits of a standardized data tape format arelost when the systems that control the data tape driveswrite custom, library-specific data to the tapes. Thislocks the facility into a single vendor’s storage libraryproduct, and makes interchange impossible with customers and other facilities that may use another vendor’s library system.

Digital VideotapeHDCAM SR and D5 are the only high-end professionalvideotape formats being used in motion picture master-ing today, although HDCAM SR is the dominant for-mat currently in use, especially for digital motion pictureacquisition. Introduced by Sony in 2003, HDCAM SRcan record HDTV images (1920 x 1080 pixels), whichis slightly less than Digital Cinema’s “2K” pixel count(2048 x 1080), and uses MPEG-4 Studio Profile imagecompression. The D5 format, introduced by Panasonicin 1995, is also an HDTV system, although the formatwas recently upgraded by Panasonic to full “2K” pixelcount and JPEG-2000 image compression, the samecompression scheme used for Digital Cinema digital“prints.” There are several technical differencesbetween the formats, but these are beyond the scope ofthis report.

There is general agreement in the industry thatthere will be little or no new development of professionaland consumer videotape formats as broadcast televisioncontinues to go “tapeless,” although that transition isnot without its digital storage issues [Kienzle, “Taking,”12]. The consensus is that although digital videotapestored in proper environmental conditions can last forat least 5 to 10 years (or longer), there may be no newvideotape format to migrate to when the medium nearsthe end of its shelf life.

Optical MediaOptical storage technology in general is not keeping upwith magnetic storage technology in terms of areal density,capacity per unit, or transfer rates. Optical disk is primarilya consumer technology, so cost per bit is very inexpensive– much lower than magnetic disk or data tape. But therate of progress of new optical storage technologies is actually slower than that of magnetic tape and disk becauseof the need for broad standards to insure interoperabilityamong many vendors’ products, and because consumersare reluctant to commit to any technology if they think itis likely to become obsolete in just a few years.


AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T

5.1 Digital Storage Technology continued

In the motion picture industry, recordable DVDs(DVD-R) are currently preferred over rewriteableMagneto-Optical (MO) disks because they are lessexpensive and have higher capacity per unit. DVD-Roffers an attractive cost per unit, but the relatively smallcapacity per unit of optical disks – between 4.7 and 8.5gigabytes, depending on how they are used – relative todata tape that holds 400 to 800 gigabytes per cartridge,is a disadvantage for handling the large amounts of datagenerated in digital motion picture production.

The new generation blue-laser DVDs, capable ofholding 35 to 50 gigabytes per disk, still have a muchsmaller capacity per unit than LTO4 tape cartridge.The related Ultra Density Optical (UDO) format,which uses a different recording technology than DVD,is capable of holding 30 gigabytes per disk cartridgetoday, and is expected to grow to 120GB by 2008,according to the manufacturer.

As a packaged media technology primarily target-ing the large consumer market, the blue-laser DVD formats must fully standardize all aspects of their technology and stabilize as a storage medium in orderto attract both content publishers and consumers in sig-nificant numbers to become profitable. This is not thecase with magnetic data tape or magnetic hard drives,where technology advancement is continuing unabatedand vendors win market share by being the first to offerhigher speeds and higher capacities per unit.

WORM capability is one of the vaunted advan-tages of optical storage because with optical WORMthere is no fear of electromagnetic interference (EMI)or accidental erasure. These are not particularly high-priority risks in most modern digital archivingapplications except when the preservation of unalteredoriginal data is legally mandated, as it is in so-called“compliance archiving” per the Sarbanes-Oxley rulesdiscussed earlier. Magnetic tape and disk products withfirmware-based WORM capabilities are also beingintroduced, which will further deflate the advantage ofoptical WORM for many users.

It has been difficult for users to get an unbiasedforecast for the longevity of optical storage media. SoNIST, together with the Library of Congress and withthe support of the Optical Storage TechnologyAssociation, spent two years testing DVD-R disks (aswell as the DVD’s lower-capacity predecessor, theCompact Disc, or CD) from multiple manufacturerson multiple playback devices to understand their lifeexpectancy characteristics. The study found that, gen-erally speaking, both DVD-R and CD-R can be verystable, maintaining data availability for tens of years,although the measured data indicate that CD-R has a

much better life expectancy compared to DVD-R:100% of CDs tested have a life expectancy of over 15 years compared to 66% of DVDs with that lifeexpectancy. The study also found that it is very difficult for users to identify which media on the market have better stability characteristics. The use ofgold in a disk’s recording layer significantly extends itslife, but it also makes the disk five times as expensive asnon-gold disks, which stifles market demand. Slowingthe write speed or using stronger lasers for writingcould also potentially increase the life expectancy ofDVD-R, but this is considered unlikely, given the overwhelming pressure to reduce costs in order to grow consumer market share.

For all the reasons described, no large archives areknown to use CD or DVD as their primary archivalstorage media. However, writeable CD and DVD arestill widely used as non-permanent transfer and deliveryvehicles for smallish amounts of digital media such assound elements, photographs and oral histories.Furthermore, pressed CDs and DVDs are often sub-mitted to the Library of Congress to satisfy copyrightand mandatory deposit requirements. But this is not tosay that optical storage technologies will never beadopted for large archival storage systems. At least onecompany has spent more than 12 years pioneering optical holographic storage, a fundamentally new tech-nology with substantially greater density and theoreticallylonger life expectancy than CD or DVD. For archivalapplications, one of the potential advantages of holo-graphic optical technology is that it is predicted to have50-year longevity, based on the manufacturer’s acceler-ated aging tests. Potentially, archives implementedwith holographic optical storage will need to migratedata “only” every 20 years or so to accommodatechanges in computer operating systems, file formatsand application software. This is much less frequentlythan is currently recommended for magnetic tapearchives, although due to the novelty of holographicstorage technology, even its manufacturer’s executivesare hesitant to recommend it as a primary archival format. They agree that technology choices for archiving must be conservative and give priority toproven reliability and multi-vendor support.

As a point of reference, the LTO tape consortiumclaims their product has a life expectancy of 30 yearsbased on accelerated aging tests. The National MediaLab has also estimated a 30-year life expectancy for magnetic tape based on its own testing [Van Bogart 34].Nonetheless, leading tape vendors and even NARA recommend data migration of digital assets on magnetic tape as frequently as every 5 to 10 years.

AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T5.2 Risks and Threats to Digital Data

TWO QUESTIONS ARE KEY TO UNDERSTANDINGwhy digital archives cannot be preserved over the longterm using a “store and ignore” management philosophy:“Is there any way to store a digital object for 100 yearswith no maintenance?” Secondly, “Is the bit densityenough to hold what you want to preserve at a price youcan afford?”

If one could make a “black box” with even 100-yearlifespan components that could read data reliably withoutintroducing any errors, required no maintenance, andoffered sufficient bit density at an affordable price, every-one would buy it. After filling the black box with theirmost valuable “permanent” assets, one of the first thingsprudent archivists would do is create several replicas in multiple black boxes and geographically separate themto guarantee viability and enable the archive to becomeself-healing. If the archive format preserved both bits andneeded application software together with contextualmetadata, there would be no need for periodic data migra-tion or system emulation. But there’s a new danger inher-ent in this approach. If the 100-year-lifespan “box” fails at99 years, no one involved in its development or capable ofsystem repair is likely to remain alive. To avoid this risk, itwould be necessary to continuously audit the integrity of

the box to ensure that the archived assets can move to anew box before the old box fails. This points to the needto sustain a supportive human community around a digitalarchive with the requisite know-how in order to ensure itsability to preserve, renew and repair the system withinwhich digital assets are stored.

Digital assets in the real world are not kept in “blackboxes” with 100-year longevity. They are stored on physical media with longevities of 30 years or less, and arevulnerable to heat, humidity, static electricity and electro-magnetic fields. The digital contents can be degraded byaccumulating unnoticed statistically occurring “natural”errors, by corruption induced by processing or communi-cation errors, or by malicious viruses or human action.Digital media cannot be viewed with the naked eye. Assuch, it is susceptible to misidentification, frequentlypoorly described (incomplete labeling and metadata), andtherefore difficult to track. And digital assets are hard tomaintain long-term because media, hardware and softwarecan all become obsolete. This is commonly caused by theevolutionary loss of compatibility between data in thearchive and the software applications that originally created the data. Sometimes proprietary formats in anarchive are simply abandoned when a company goes out

TOTAL ANNUAL COST PER TERABYTE = $1500


3 5 YEARS

5 10+ YEARS

3 5 YEARS

.5 10 YEARS

VARIES

VARIES

LIFESPAN HARDWARE SOFTWARE

HOST COMPUTER

MEDIA

TRAINED PERSONNEL

FUNDING

• APPLICATION SOFTWARE • OPERATING SYSTEM • DEVICE DRIVERS

PHYSICAL INTERFACE • INTERFACE FIRMWARE

MEDIA DRIVE • DRIVE CONTROL FIRMWARE

• FILE SYSTEM

• DATA FILE FORMAT • PHYSICAL RECORDING FORMAT

Digital Archive Layers



AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T

5.2 Risks and Threats to Digital Data continued

of business. A digital archive may have many "layers,"each with its own finite lifespan as shown in the diagram on the facing page. When the end of the lifespan is reached, not only does the layer have to bereplaced, but the adjacent layers may have to be modified to be compatible with the replacement layer.Thus, a digital archive built with today’s digital tech-nologies can only assure digital “permanence” via anongoing and systematic preservation process.

The rapid and seemingly endless improvement inthe price per bit of digital data storage tends to give theimpression that storage is forever getting cheaper, sowhy worry about the “data explosion”? There are severalreasons why the overall storage picture is not as simpleas this might make it seem:

Increasing demand for storage offsets reduced media costAlong with the increase in available storage comes a corre-sponding increase in the demand for storage. In the UCBerkeley digital data generation study discussed earlier, itwas found that of the 5 exabytes of new data created in2002, 92% was recorded on magnetic media, 7% on film,and the remaining 1% split between paper and opticalmedia. Overall, UCB researchers estimated that newstored information grew about 30% from 1999 to 2002.

From the relatively narrow view of the motion picture industry, one only need consider the amount ofdata generated by the new generation of 4K digital motionpicture cameras and digital postproduction process (in thepetabyte range) to understand that there will always be away to generate more data, usually in excess of availablestorage. The demand is compounded by the need toduplicate important data for backup purposes.

Data transfer rates do not increase at the same rate as storage densityAs the storage density grows, the speed at which thedata gets on and off the storage media (transfer rate, orthroughput) becomes more important. The need forincreased throughput drives up the cost of the physicalinterface, network connections and computers attachedto the disk drives. As with the demand for increasedstorage, throughput requirements increase with theneed to make backup copies of important data.

Longevity characteristics do not always meetadvertised specificationsRecent studies by Google [Pinheiro] and the ComputerScience Department at Carnegie Mellon University

[Schroeder and Gibson 15] present evidence that harddrives are not as reliable as manufacturers’ data sheetssuggest, nor do they follow the conventionally accepted“bathtub curve”9 failure characteristic. To the contrary,these studies observe that large numbers of drives failwell before manufacturer-specified “mean time beforefailure” (MTBF), and show a low correlation betweendrive failure rates and high temperatures, a commonlyassumed failure predictor.

The manager of a large digital image archive inter-viewed for this report who has purchased a great deal ofboth tape and disk over the years said that in his experi-ence, the biggest problem with a magnetic hard drive is its short device life cycle, supposedly five years accordingto manufacturers, but only three years in practice. Herecognizes that disk technology is driven by personalcomputing and consumer electronics markets character-ized by very short product life cycles, so there is naturallyquite a bit of product churn. In contrast, data tape drivesare industrial products, with multi-year life cycles, andwith some degree of backward compatibility and forward-looking roadmaps from vendors.

These empirical observations raise questions aboutthe “accelerated age testing” methodologies used by storage product manufacturers to determine the lifeexpectancy of their products, and suggest that there is noway of knowing whether a storage device or medium will,on average, last for the advertised period of time withoutactually seeing what happens during that entire timeframe. It is worth repeating that both storage technologysuppliers and end-users significantly de-rate the publishedlife expectancies of all digital storage systems, usuallyplanning on wholesale equipment and media replacementafter as little as three years, with five to ten years as themost often quoted migration period.

Economic, Technical and Human ThreatsA recent report by the National Research Council written for the National Archives [Natl. Research 59-69], presents the notion of threat modeling and threatcountering as a core consideration in the design of digitalpreservation systems. These threats are further detailed ina paper on digital preservation system requirements pub-lished by the Stanford University Libraries [Rosenthal 3],and are worth summarizing here for the benefit of those responsible for preserving digital motion picture assets:

Economic threat:

• Funding loss: Digital preservation systems require ongoing funding for equipment maintenance,

9 The bathtub curve, used in reliability engineering, predicts early “infant mortality” failures, followed by a constant failure rate during a product’s useful life, followed by an increasing failure rate. A graphed curve of these failure characteristics looks like a bathtub – high at the ends and low in the middle.

AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T5.2 Risks and Threats to Digital Data continued


replacement, operating staff and power, among other things. Every commercial enterprise has its good and less-than-good years, and the occasional “benign neglect” that film archives can tolerate may result in data loss in a digital archive. There is no known tactic to fully mitigate this threat, although factors that affect the economy of operating a digital storage system are discussed in Section 6.

Technical threats:

• Data integrity: At the most basic level, the 0s and 1s that represent digital images and sound must be reliably stored and retrieved. Common failure modes that affect the integrity of the 0s and 1s preserved in digital archives are latent errors (errors lurking undetected), ingest errors (translation errors when digital data is brought into a digital system), and network communication errors (errors caused when digital data is moved between computers on a network). Regular auditing and authentication of the data and rigorous quality control procedures are effective means for dealing with these threats.

• Monoculture vulnerabilities: Just as a single animal species can be wiped out from a deadly virus, individual storage media or technologies can be (and have been) seriously impacted in the same way [Herman]. Biodiversity, or the practice of utilizing several different media and technologies for digital storage, significantly reduces this threat [Baker 8; Science Applications 31].

• Single point-of-failure: Storing a single copy of data in just one location is dangerous. Storage solutions should include sufficient redundancy to protect from data loss resultingfrom the failure of media, hardware, software, network services and/or natural disasters.

• Obsolescence: All of today’s technology products, including storage media, hardware and software, have a finite lifetime, and the time required to migrate can exceed the data’s lifetime.

• Limited or no data compression: A popular technique for reducing storage and transmission bandwidth needs is to apply mathematical data reduction techniques to image and sound data. These techniques range from “mathematically lossless” (every single bit is recovered when decompressed), to “perceptually lossless” (not every bit is recovered, but one cannot see or hear the difference between the decompressed content and the original), to “lossy” (perceptual artifacts exist in the decompressed content). The effects of compression must be well understood if used.

• No risk of encryption key loss: There is much discussion today on safeguarding digital content through the use of data encryption methods. All encryption schemes require a digital key to “unlock” the encrypted content. If encryption is deemed necessary, then steps must be taken to eliminate the risk of losing the key, which is tantamount to losing the content it is intended to unlock. In general, there is broad consensus among those interviewed for this report that encrypting digital archives increases long-term complexity and risk.

Human threats:

• Operator error/malicious action: Today’s technology requires human involvement in many aspects of digital storage system operations. And being human means mistakes can and will be made. Furthermore, systems can be attacked by disgruntled employees or hackerssimply doing it for fun. Procedures for protecting against losing media, unauthorized internal and external system access, reliance on a single employee, and storing multiple copies of important data in separate locations not controlled from a single place can be effective in managing the human element. Documentation of procedures and system implementation details can also protect against organizational failures that often occur when companies are sold or merged, or when key employees move on.

All of today'stechnologyproducts,including storagemedia, hardware andsoftware,have a finitelifetime, andthe timerequired tomigrate canexceed thedata's lifetime.


AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T

5.3 Digital Preservation Strategies

BROADLY SPEAKING, DIGITAL ARCHIVINGexperts have identified several preservation strategiesthat address either the general survivability of digitaldata or technical obsolescence. Two of those strategiesare discussed here: migration and emulation.

Migration Data migration involves the transfer of data from oldphysical media to new physical media, a process thatoften (but not always) includes updating file formats forcurrency with the latest-generation operating systemand/or software applications. Older digital assets that areproperly migrated will be accessible for some time intothe future, until technological obsolescence motivatesanother migration cycle. Migration is designed to avoidhaving to preserve old devices to read the old storagemedia, old application software to interpret the old data,and old hardware to run the old software to use the olddata. If everything goes smoothly, after migration thenew data replaces the old data.

A major drawback to migration is that while copyingdata from one physical medium to another, or while con-verting digital assets from one file format to another,some data (or related metadata) might be lost. To makedata migration a lossless, errorless process, migration procedures typically incorporate various quality controland auditing routines to ensure accuracy, integrity andcompleteness of the data throughout the migrationprocess. Systemization of the migration process, includ-ing policy-driven automation routines, reportedly can beeffective in reducing human errors and increasing thespeed of migration. In practice, the emerging trend is to“migrate all the time” as a background task.

Migration of archived assets by replicating them onnew media is a preservation strategy for both analog anddigital assets. An advantage of migration as a digitalpreservation strategy is that digital assets will always beavailable in the form that is most widely accepted, andcurrent hardware and software will be able to render thesedigital assets with little difficulty. In the case of analogassets, migration can cause the loss of image and soundquality over successive generations. In the case of digitalarchiving, data migration done correctly is lossless every

time. Data migration can occur between instances of the same type of storage medium, from one medium toanother, and from one format to another. Data migration can be effective against media and hardwarefailures. For example, the tape backup of the contents ofa magnetic hard drive involves data migration betweendifferent mediums.

The goal of archival data migration is preservation ofthe full information content, not just the bits. For exam-ple, the Open Archival Information System (OAIS), pioneered by NASA and others, defines “preservationdescription information” that should be included in thedata migration process. This includes provenance infor-mation that describes the source of content, who has hadcustody of it, its history, how the content relates to otherinformation outside the archive, and fixity informationthat protects the content from undocumented alteration.

Data migration can be motivated by a variety of factors such as physical media decay, media or mediadrive obsolescence, even prior to complete system obsolescence. Older media drives may face escalatingmaintenance costs, there may be new user service requirements, or new media formats and/or file formatsare introduced that are more compatible with users’ technology and applications. The list of motivating factors goes on, and therefore data migration is the mostwidely practiced digital preservation strategy today.

EmulationEmulation preserves the original data format, often on theoriginal physical medium, and provides the user withtools that enable the data to be read even after the orig-inal file format, storage medium, application program orhost hardware is no longer supported. Emulation refersto the ability of one system or device to imitate anothersystem or device. In practice, emulation involves writingsoftware that runs on new hardware to make it appear asif it is an old system, translating between the two, allow-ing old data on old media to be “tricked” into workingon a new system after the old underlying system hasbecome obsolete. For example, new storage devicesadded to existing digital storage systems are often builtwith the ability to emulate an older storage device, so

Data migration is the most widely practiced digital preservation strategy today.

AR

CH

IVIN

G I

N T

HE

CH

AN

GIN

G E

NV

IRO

NM

EN

T5.3 Digital Preservation Strategies continued


that the new storage technology can be integrated intothe pre-existing software control and automation infra-structure of the system, thereby hiding the evolution ofthe infrastructure from the end-user. Emulation strate-gies for digital preservation are designed to minimizethe need to copy, transfer, transform or otherwise“update” the digital assets in an archive. Digitalarchivists can use emulation strategies to reduce or even(theoretically) eliminate data migration. However, aserious drawback to emulation is the cost and complexityof developing and maintaining emulation tools. Toavoid the risk that old emulation tools will not work onfuture computer platforms, software engineers mustkeep adapting and updating them.

While emulation has not been widely adopted asthe primary digital preservation strategy for major digital archives to date, researchers at the University ofMichigan and the University of Leeds in the UK, working with the BBC on the Domesday Project (asdiscussed earlier in this report), have demonstrated thatemulation can preserve the consumer’s experience ofinteractive multimedia based on older videodiscs andCD/DVD-ROM systems. They point to the need foremulation techniques in any effort to archive videogames and hyperlinked rich-media documents.

This has led researchers, particularly some fromIBM, to propose emulation strategies for long-termpreservation based on the concept of a “Universal VirtualComputer” (UVC), a layer of software that remains thesame on the “top side” facing the emulation tools whileevolving as needed on the “bottom side” facing the hard-ware and operating system (OS) software to adapt tochanges in technology. In this approach, digital assetdata is archived with a very basic software program thatdecodes the data and returns the asset in a readableform using a future software application based on a

logical view that is simple and self-contained enough tobe interpreted without any specific software or hard-ware. Working with the National Library of theNetherlands, IBM has successfully shown a proof-of-concept of the UVC approach using electronic documentsdeposited in the library in the Adobe Acrobat electronicdocument format [Lorie 6].

Some argue that emulation, and its distant cousinencapsulation,10 are just more complicated forms of datamigration.

No one strategy is “best”In considering emulation versus migration, expertsagree that no one strategy is ‘best” for long-term preser-vation of digital data. Both emulation and migrationhave pros and cons. In general, storage vendors havetended to promote migration, while computer and software vendors have tended to promote emulation.Some digital preservation researchers advocate a hybridapproach, combining both migration and emulation.For example, emulation uses a “root format” fromwhich digital asset transfers and conversions can be generated even as hardware and software evolve. Butsometimes new formats are just too attractive to passup, so an archive might periodically migrate its data tothe new better/faster format, which then becomes thenew root format for subsequent emulation. Amongoperators of major digital archives we interviewed,migration is the overwhelmingly preferred strategy fordigital preservation at this time. But these same expertsrecognize that emulation also has merit, and admitemulation has been under-explored as a strategy forlong-term preservation. Perhaps migration is the more conservative strategy and emulation requires higher initial investment in software development.

10 Encapsulation is another digital preservation strategy that proposes “wrapping” a digital asset with instructions on how to be decoded.

MAGNETIC HARD DISK STORAGE (1.8 PB)

TOTAL ANNUAL COST PER TERABYTE = $1,500

$1,600

$1,400

$1,200

$1,000

$800

$600

$400

$200

0$100

$100

ARCHIVAL TAPE STORAGE (5 PB)

COLOR KEY

MAINTENANCE/SYS. ADMIN. LABOR

FACILITIES COST/SPACE, UTILITIES

MAINTENANCE & LICENSE COSTS

OTHER CAPITAL COSTS (ANNUALIZED)

DATA/CARTRIDGE MEDIA COST (ANNUALIZED)



Estimated Normalized Cost of Delivering Disk and Tape Storage at SDSC

$165

$110$25

$100

$535

$235

$230

$160

$340COLOR KEY

DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S

6 Digital Motion Picture Archiving Economics

THE ECONOMIC IMPACT OF USING digital technologies in motion picture masteringand acquisition cannot be fully understood withoutan understanding of the complete costs associatedwith depending on digital storage technology forthe long-term accessibility of important digital data.

6.1 Digital Storage Economics

BASED ON CONVERSATIONS WITH SEV-eral experts in digital archiving, it is clear that the eco-nomic model for digital archiving requires reconsider-ation of basic assumptions about both the costs andrewards of preservation. The total cost of ownershipof operating a digital archive is typically expressed as$/terabyte/year. However, many vendors presenttheir competitive advantages most favorably by simplifying the cost components to just the storagemedia and devices they sell, ignoring other costs thatthe user will inevitably face. Other vendors will com-pute costs based on an archive sized to fit their tech-nology most economically. Inaccurate or incomplete

cost analysis is not limited to vendors. Several archiveoperators interviewed for this report acknowledge thatthey have either not tried to compute a full costanalysis, or tried and gave up because expense infor-mation is hard to collect for administrative reasonswhen budgets and cost accounting are project-basedand “stove-piped” due to organizational structures.

There are, however, several archival costanalyses from well-regarded organizations, breakingout all significant expense types and distinguishingbetween tape and disk storage. The San DiegoSupercomputer Center recently published a paperthat discusses a comprehensive cost model for its25-petabyte capacity mixed magnetic disk/data tapestorage system that currently holds approximately 7petabytes [Moore 2].

The chart below shows the estimated normalized annual cost of delivering disk and tapestorage at SDSC. The only costs not included are transaction costs; i.e., the cost of transmittingand receiving the data, networking/bandwidthcosts, etc. Disk drive utilization is discountedfrom 100% to account for data overhead andoperating efficiency – a consideration for anymagnetic hard drive-based system.


Estimated Normalized Cost of Delivering Disk and Tape Storage at SDSC in 2006

DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S6.1 Digital Storage Economics continued


The totalcost-of-ownershipcalculationshouldinclude the cost of datareplication– that is,multiplecopies of data forprotectionagainstloss.

TRANSACTION$/TB/YR$/TB

Amazon S3

Cleveland Clinic

SDSC

Swedish Natl. Archives

Unknown

1 PB(4 PB Capacity)

7 PB(25 PB Capacity)

40 TB(200 TB Capacity)

Representative Annual Total Cost of Ownership

Hard drive

Mixed tape/hard drive

Mixed tape/hard drive

Tape

$1,843

$1,500

$500 – $1,500

$11,344

$133 – $184

Unknown

Unknown

Unknown

ARCHIVE SIZE STORAGE TYPESOURCE

It is interesting to note that storage media costs are only 36% and 20% of thetotal annual operating costs for magnetic hard drive and data tape systems, respectively.It is also interesting to note that although data tape is one-fifth the cost of hard diskson a cost-per-bit basis, the annual expense for a large data tape storage system is one-third the cost of a hard disk-based system.

The numbers appear to scale in inverse proportion to the size of the storage system. The Cleveland Clinic reports its costs at $1,500/terabyte/year for a 1-petabytemixed tape/disk system, and the Swedish National Archives’ projected annual costover five years for its 200 terabyte-capacity data tape storage system (less than1/100th the capacity of the SDSC tape system) is over $11,000/terabyte/year, a 7-fold increase over the SDSC annual costs11 [Palm 7].

As another reference point, Amazon.com, the large online retailer, recently intro-duced an online storage service called Simple Storage Service, or S3, targeted at soft-ware developers. Customers can upload data to the S3 service and pay a monthlystorage fee of $0.15/month/gigabyte plus a transaction fee of $.10/gigabyte for datauploading and between $0.13 and $0.18/gigabyte for data access, depending on vol-ume. This translates to $1,843/terabyte/year for data storage services, plus $102 perterabyte for initial data upload, plus between $133 and $184 per terabyte per access.

The ultimate total cost-of-ownership calculation should include the cost of datareplication; that is, multiple copies of data for protection against loss. For example,one hard drive copy and one tape copy at SDSC would be $2,000/terabyte/year.The S3 system, according to Amazon.com, replicates individual data objects acrossmultiple storage “nodes” and physical locations, with at least two copies of dataobjects in existence at any one time.

Looking to the future, the SDSC study states that the cost differential betweenmagnetic hard drive and data tape storage will likely diminish. One of the study’sauthors observed that data tape media costs are falling by half every 36 months, magnetic disk media costs are falling by half every 15 months, and a “crossover” inthe two media costs is forecast in 2009-2010. But even if magnetic disk media costsless than data tape, it may be that data tape will remain best for “cold storage,” giventhe higher power consumption per terabyte of magnetic hard disk storage and theexpected long-term increase in the cost of electrical power.

11 The Swedish National Archives numbers were calculated in 2005 and the SDSC numbers were calculated in 2007, so the relative difference might be somewhat less due to ongoing storage media cost-per-bit improvements.


DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S

6.2 Digital Motion Picture Storage Economics

IT IS IMPORTANT TO DETERMINE THE COSTof archiving suitable elements for preserving and creatingmotion picture content well into the future. These elements are generally considered to be the mastermaterials from which all downstream distributionmaterials are spawned, with an expected access time-frame of at least 100 years. This section of the reportapplies what was learned about the total costs of digitalstorage to the digital elements produced in typicalmotion picture productions.

To develop an understanding of the actual motionpicture deliverables requiring long-term storage, twocase studies were undertaken based on actual recentmotion picture productions. One case study was amotion picture captured on film and digitally finishedto distribution. The second case study was a motionpicture that was captured digitally, or “born digital,”and also finished digitally through to distribution. Thescope of this analysis was limited to picture and soundelements created during production and postproduc-tion that led to worldwide theatrical exhibition.

The film-capture production was chosen fromamong a number of average-length features (90 to 120minutes) with an “average” budget (>$60 million) and few if any visual effects. The digital-capture production also met the same content criteria and was photographed using a modern digital motion picture camera generating digital frames at 1920 x 1080 pixelcount. The camera output was recorded to HDCAMSR videotape (this case study preceded digital capturedirectly to magnetic hard disk), and the digital “mas-ters” were also created at a pixel count of 1920 x 1080and stored on magnetic hard drives and LTO data tape.The studios participating in the case studies providedcomplete inventory reports for each feature.

Each studio uses proprietary software systems totrack archive and library assets. Since the two casestudies contained inventory data from different studios,there was a need for a common reference of generatedmaterials. This was accomplished by creating a generichierarchy of materials for both picture and sound,which was then populated with elements described inthe separate inventory data supplied by the participat-ing studios. These picture and sound hierarchy chartsare in the Appendix. The charts are color-coded toindicate to which storage category (archival or workinglibrary) each element is assigned. The source informa-tion for the hierarchy charts is an amalgamated (albeittypical) delivery schedule used by studios in third-partyproduction agreements. The delivery schedule is a legalapproach to describing the known “common sense”results of the production process and specifying howthe studio expects to receive these items at the point of

final delivery. A successful delivery is usually tied tothe final payment, so producers are keen to understandthe exact delivery requirements from the studio as earlyin the process as possible. In the end, a productioncompany will deliver a mountain of film, paper, mag-netic hard drives, DVDs, data tape and videotapesaccording to the schedule.

Executives from participating studios wereextremely helpful in validating the accuracy of theresults. One studio opened several cartons of aggregat-ed materials to provide an average count of like mediafor the analysis. This average count is conservative, andit is used in the digital capture case study to estimatethe average number of HDCAM SR camera originalvideotapes that are stored in a single carton.

The case study information is presented in a seriesof tables in the Appendix that summarize the numberand type of elements, and their estimated annual stor-age costs. Because of varying practices between studiosand production workflows, several assumptions werenecessary regarding the calculation of the number ofelements and the “byte count” of the digital elements.As with any case study, the results represent a snapshotof time, and while production practices continue toevolve, the data presented are still considered valid atthis time, although the summary data presented in thissection incorporates further assumptions (describedlater in this section) to reflect current industry trends.

In the summary cost analysis, two key definitionsare used:

• “Archival” is defined as storage of the master elements from which all downstream distribution materials can be created over a 100-year timeframe.

• “Working Library” storage is a broad term for elements that are generally kept on hand for distribution purposes.

The participating studios save their elements ineither archival storage conditions or working libraryconditions, depending on their preservation and near-term access policies.

The only picture element that continues to achievebroad consensus as the indisputable archival master picture element for a major motion picture is the YCMseparation master on black-and-white polyester filmstock. The current cost of creating a complete set ofarchival separation masters is estimated to be between$65,000 and $85,000, depending on service options.

With respect to base storage costs, physical elementstorage cost information was obtained from severalcompanies engaged in that business, given the difficultyin determining accurate on-the-studio-lot storage costs.

6.2 Digital Motion Picture Storage Economics continued


The cost figures for data storage came from the San Diego Supercomputer Centerstudy discussed earlier in this section, which describes the lowest observed cost for afully managed digital storage system with both online magnetic hard drive storage andnear-line data tape storage. It is important to restate that this baseline cost representsonly a single fully managed copy of the data.

The baseline storage costs used for this study are:

• $4.80 per physical item per year for archival storage

• $1.80 per physical item per year for working library

• $500 per terabyte per year for near-line data tape storage

Initial inspection and access costs are not included in the baseline film storage costs, nor are access or ingest costs included in the baseline digital storage costs becausereliable information for the latter is not available. Nonetheless, these costs should betaken into account when considering the type and quantity of assets being stored.

The table on the facing page summarizes the annual storage costs, exclusive of ingest, inspection and access costs, for five common scenarios:

1. An “all film” production that generates no digital assets

2. A film-captured, digitally finished production at 4K

3. A digitally captured, digitally finished production using HDCAM SR videotape as the capture medium at 1920 x 1080

4. A digitally captured, digitally finished production using an uncompressed digital data capture system at 2K

5. A digitally captured, digitally finished production using an uncompressed digital data capture system at 4K

The film-capture/digital finish production and 4K-captured productions produce4K masters, and the 2K-captured productions produce 2K masters.12 Three copies ofthe digital master are assumed, given the recommended practice of data replication,and this is consistent with the practice of archiving between two and five film masters,although three is most typical: a YCM, a finished negative, and an interpositive. Thecost of producing the three film masters ($80,000 amortized over 100 years) is addedto the annual storage cost of film. The finished master is assumed to be 120 minutesin duration for all scenarios; a shooting ratio of 25:1 is assumed as an industry averageto calculate the amount of source material, and two copies of all digital source materialis assumed to reflect current industry practice and insurance requirements.

Using current preservation methodology, the cost of storing 4K digital masters was found to be enormously higher – 1,100% higher – than the cost of storing filmmasters. The overall costs increase further with the use of magnetic hard drive datacapture systems at 2K, and further still with 4K digital capture systems.

Although 1920 x 1080 capture using HDCAM SR videotape appears to be a cost-effective alternative to 4K, it is worth repeating that this cost reduction comes with acorresponding reduction in certain performance characteristics relative to film. These

The cost of storing 4K digitalmasters wasfound to beenormouslyhigher –1,100% higher – thanthe cost ofstoring filmmasters.

12 For this report’s calculations, a 4K frame is composed of 4096 x 2160 pixels, 48 bits per pixel; a 2K frame is composed of 2048 x 1080 pixels, 30 bits per pixel; and a 1920 x 1080 frame is composed of 1920 x 1080 pixels, 30 bits per pixel.

DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S


DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S

6.2 Digital Motion Picture Storage Economics continued

performance characteristics, and their effect on per-ceived image quality, are beyond the scope of thisreport, although there is significant and ongoing debateabout these tradeoffs. Furthermore, the decision tomigrate HDCAM SR source material would likely bemade in approximately 10 years.13 If the choice is madeto copy the tapes to some newer videotape format(assuming such a videotape format is developed in thefuture), the cost to do so is estimated as follows:

• Total number of original production videotapes(from case study): 5,347

• Cost of new tape stock: $100 per tape

• Cost of copying to new videotape: $400 per tape

• Total cost of migration: 5,347 x ($100 + $400) =$2,673,50014

Again, there are longer-term costs to considerbeyond those associated with the initial creation of adigital motion picture.

With an understanding of the new storage costrealities of digital motion picture data, the questionsthat must be asked now include: What materials shouldbe stored for commercial exploitation on some new distribution technology or platform as yet unseen?What bonus materials will be needed? What about apotential “director’s cut” or newly edited version? Is itsufficient to protect only the finished master? Should“mild” or “mathematically lossless” data compressionbe considered to reduce digital storage requirements byone-half or more? Is the image quality of the archivedmaster sufficient for future display technologies? Thenew economics of digital motion pictures require acareful look at the complete asset picture.

ARCHIVAL MASTER SOURCE MATERIAL

Annual Storage Costs of Motion Picture Materials

$1,059

$1955

$1,955

$486

$208,569

$486$180

$32,589

$220,000

$210,000

$200,000

$50,000

$40,000

$30,000

$20,000

$10,000

$1,000

0

$12,514

$1,830

$12,514


COLOR KEY

ALL FILM

FILM CAPTURE, 4K MASTER

DIGITAL CAPTURE TO HDCAM SR TAPE,1920 X 1080 MASTER

DIGITAL CAPTURE TO 2K DATA, 2K MASTER

DIGITAL CAPTURE TO 4K DATA, 4K MASTER

Annual Storage Costs of Motion Picture Materials

13 The perceived longer “shelf life” of videotape as compared to data tape is attributed to the (generally) longer useful life of videotape formats, lack of dependence on computer hardware, operating systems and application software, and the use of error concealment techniques to counter increasing bit error rates over time.

14Tape stock and copy costs based on current high volume dubbing rates for HDCAM SR tapes.

DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S6.3 What This Means for the Motion Picture Industry


6.3.1 Economics of Archiving are ChangingTRADITIONAL ANALOG ARCHIVING COSTstructures have high initial delivery and archive accession costs, followed by low storage maintenanceexpenses until such time as the analog asset needs to beaccessed and utilized, when there may be substantialadditional costs. On the other hand, digital archivingof born-digital assets has lower initial delivery andaccession costs and lower costs associated with accessand utilization of the asset, but requires higher levels ofinvestment to support the ongoing digital preservationprocess which may include digital migration. Thisincreases the importance of organizational continuityand sustained funding. To date, storage (disks andtape) has been the biggest expense category, but as seenat the San Diego Supercomputer Center, EROS, andSwedish National Archives, as digital archives scale upand storage hardware prices decline over time, theongoing costs of storage technology trend down whilethe costs of data management services, labor and powerincrease as a percentage of the total cost of ownership.

Traditional archiving is widely accepted as justanother “cost of doing business” in Hollywood and else-where. For example, in every county in America (andother countries, no doubt) the tax assessor’s office isresponsible for maintaining accurate, up-to-date recordsof property ownership, property transfers, property definitions (surveyed plot lines) and taxes paid everystep of the way. These archives are continuously growing. The assessor’s archiving policy must be “saveeverything” because new records point to old recordsfor authenticity and to describe changes. Records arenever purged. This costs money – always has, alwayswill. But the archiving of property ownership and taxpayment records is a cost that society accepts as a necessary fact of life.

The economic model for traditional film archivesincurs most of the expenses up front, in the form ofone-time costs to acquire the collection, one-time con-struction costs for the building to hold the collection,and smaller ongoing expenditures for labor and powerneeded to catalog, copy and preserve the collection.There are also typically variable expenses to access/con-

vert or restore film assets when they are retrieved, inorder to make them useable/saleable. To reiterate, thelongevity of film archives is primarily determined bymedia durability and proper use of media-specific conservation techniques, and secondarily by sustainedfunding and organizational continuity. Specializedskills required by traditional archivists are well-estab-lished organizational and managerial competence and a range of “white-glove” conservation techniques forspecific media types.

Both traditional and digital archiving generallyrequire investment “today” in the belief that some benefit will be realized in the indefinite “tomorrow.”Historically, most archives have been operated as non-profit “public works” for religious or scholarly purposes,or the good of society as a whole. Many cinemaarchives around the world continue to operate as publicarchives. Digital conversion can expand potential accessfor future generations, conveying the cultural “patrimony”to more citizens. In Hollywood, private cinemaarchives have developed as valuable corporate assets thatappreciate over time and can yield profitable commer-cial media products in the future. Digital archiving willenhance the potential for commercial exploitation ofHollywood’s media assets and will play an increasinglycentral role in the business. But private digital cinemaarchives are not going to be cheap, they are not goingto (immediately) eliminate the old costs of film archiving,and they require a new business model to sustain digitalpreservation activities.

As explained elsewhere in this report, longevity ofthe digital archive using current technology and proce-dures is primarily determined by digital migration oremulation rather than physical conservation of mediaobjects. So digital archives will require recurring expen-ditures to support the regular procedures of data auditand exercise, data quality control, data migration and/oremulation required for long-term preservation of digitalassets. These processes can be done as periodic batchjobs, which leads to peaks and valleys in the workloadof the staff, bandwidth requirements for the system,operating expenses and capital investment, all dependingon the periodic frequency of technology obsolescence-driven data migration. But the more modern approach

The ongoing costs of storage technology trend down while the costs of data management services, labor and powerincrease as a percentage of the total cost of ownership.


DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

Sat larger digital archives has been to automate datamigration so it can occur as an ongoing back-ground task in order to smooth out the workloadand the budgetary expenses over time. Thisincreases the importance of organizational conti-nuity and sustained funding.

At the same time as the cost structures ofarchiving are changing with the transition fromanalog to digital, it is clear that access to archives(even analog/film) has become increasingly valu-able over the past few decades. Digital archivesare potentially more accessible than analog/film,which implies that digital archives will becomepotentially more valuable than analog archives.Digital archives offer the benefits, compared toanalog archives, of faster search and asset retrieval,easier local and remote access via networks, lowercost of replication and distribution, and easier andfaster format conversion, including the ability toextensively “slice and dice” old digital assets intonew content that can be commercially exploitedfor new markets via new distribution channels.

6.3.2 Save EverythingCURRENT PRACTICE IN HOLLYWOOD ISto “save everything” on film in the film archives orwarehouse storage facilities. This ensures futureusers will have maximum flexibility to pick andchose what they want, when they want it. As anarchiving policy, “save everything” is fast, compre-hensive and simple to understand. And it is safe,because no one has to take responsibility for deciding what not to save. This practice has beenextended to include most every element not onfilm, which includes paper documentation, digitaldata tapes, videotapes, optical disks and hard drives.

At one studio, a senior technologist recog-nized the potential for future value of all assets,but said the decisive reason to save everything isthat it is just too troublesome to sort out thedesirable materials from the undesirable. The eas-iest thing to do is throw it all in the vault. And,in the view of this technologist, ideally the digital“vault” should be online magnetic disk storagebecause this will make the digital assets moreaccessible for both re-purposing and data mining,including new techniques to extract contextualand descriptive metadata automatically.

Historically, motion picture elements wereand continue to be stored in several locations: onthe lot in the studio archives, at independent filmarchives and storage warehouses, and in many

cases, at film labs and postproduction houses atno cost as a courtesy to their studio clients. This model has well served the studios when theelements are completely film-based, but the situation changes dramatically when digital elements are involved.

6.3.3 Don’t Save EverythingTHE “SAVE EVERYTHING” POLICY,whether motivated by concerns about future salesopportunities or adopted because it is the path ofleast resistance, must confront the practical realitydescribed by several studio executives, which isthat studios are producing so many bits and piecesof digital content that they cannot afford to saveeverything forever, but instead must learn what todiscard. This is especially true for feature-lengthmotion pictures because, as we have seen, long-term digital archiving incurs ongoing preservationcosts that are significantly higher than archivingfilm – on an annual basis, $8.83 per runningminute to archive a film master versus $104.2815

per running minute to archive a 4K digital master.It is useful to look again at what is being

done in television. ESPN – arguably the largestcablecaster of sporting events – is awash eachweekend in data tapes. ESPN covers professional,college and local sports in most categories, and byMonday morning, unless the weekend’s capturedmaterial is cleaned out, there may be no physicalroom to ingest the next week’s national and international feeds. ESPN’s guiding rule is to save only assets that cannot be reasonably recon-structed. In their case, the sheer volume of dataforces decisions on what to save, and the samesheer volume of data makes it impossible to saveeverything. The decision as to what to discardand what to save has been described as “triage onthe fly.” In the motion picture industry, the practice of deleting digital non-“circle takes” onset has been reported, but until the cost of digital archiving is considered, there has been no compelling reason to do this or some otherculling process on a regular basis.

6.3.4 Who Decides, How, and When?THE VALUE PROPOSITION OF “SAVEeverything” is changing as the media business goesdigital because for the first time in history it isbecoming feasible to create digital distributionlibraries and digital archives capable of exploiting

15 Assumes 3 copies of digital data and a YCM separation/negative/interpositive set for film.

6.3 What This Means for the Motion Picture Industry continued

DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

S6.3 What This Means for the

Motion Picture Industry continued


the marketing theory of “The Long Tail.” While theeconomic aspects of this theory present an interestingmarketing concept, its application and attendant cost ofsaving everything for future and unknown uses are notnecessarily practical for today’s motion picture contentowners. This theory came up in several interviews forthis report, and therefore deserves some discussion.

The Long Tail theory was first articulated in 2004by Chris Anderson of Wired magazine to explain changing sales trends for digital media over the Internet.Anderson argued that products that are in low demand orhave low sales volume can collectively make up a marketshare that rivals or exceeds the relatively few current bestsellers and blockbusters, if the store or distributionchannel is large enough. The Long Tail acknowledgesthat sales volumes for a unit of digital media are highestat the time of initial release due both to novelty and

promotional activities typical of the modern “hits” mediabusiness. After the initial period, sales derived from adigital media asset will decline, just as for traditionalpackaged media. But the Long Tail theory asserts thatthe low cost of delivery (but not of storage and access) of digital media over the Internet allows customers tocontinue buying a given title for a longer period with little incremental expense to the owner of the media assetbecause no physical duplication or delivery is required tocomplete a profitable transaction. The Long Tail marketextends past the initial peak of revenue and subsequentdecline, to a new, third phase of commercialization whenthe value of digital assets is extended because of rarity ornostalgia or unexpected opportunities to re-purposeassets for new distribution channels. Long Tail theoristsargue that digital media and digital distribution willlead global media companies to expand their business

TIME (YEARS)

$100

THE LONG TAIL

Content Lifetime

0 1 2 3 4 5 6 7 8 9 10 100

FILM LIFETIME

THEATRICAL RELEASE

NON-THEATRICAL RELEASE

DATA MIGRATION

REV

ENU

E ($

)

Content Lifetime


DIG

ITA

L M

OT

ION

PIC

TU

RE

AR

CH

IVIN

G E

CO

NO

MIC

Sby first selling a few products in large volumes to massaudiences, and then selling small volumes of manyproducts to thousands of niche markets, reaching newcustomers who are willing to dig deeply into a studio’smedia catalog.

The key supply-side factor that determines whethera sales distribution has a Long Tail is, of course, thecost of inventory storage and distribution. Whereinventory storage and distribution costs are insignifi-cant, it becomes economically viable to sell relativelyunpopular products; however, when storage and distri-bution costs are high, only the most popular productscan be sold.

Netflix is often referenced as a Long Tail businesssuccess story. A traditional movie rental store has limitedshelf space, for which it pays facilities overhead; to max-imize its profits, it must stock only the most popularmovies to ensure that no shelf space is wasted. ButNetflix stocks its movies in centralized warehouses, so its storage costs per unit are far lower and its distri-bution costs are the same for a popular or unpopularmovie. Netflix is therefore able to build a viable businessstocking a far wider range of movies than a traditionalmovie rental store. Those economics of storage anddistribution then enable the advantageous use of theLong Tail. Reportedly, Netflix finds that in aggregateover time, “unpopular” movies are rented more thanpopular ones. And the people watching these sorts ofmovies are typically prepared to accept a delay of a fewdays between requesting a title and watching it.

The Long Tail theory of digital media marketing isconsistent with new “data mining” practices emergingin other fields such as oil/gas exploration, medicalimaging, Earth observation science, and even commer-cial credit card services. Data mining essentiallyinvolves analysis of old data (from the digital archives)using new algorithms running on more powerful com-puters than were available when the data was originallygenerated in order to extract new value from the olddata. Data mining is being successfully used to findnew oil and gas deposits, perform epidemiological studies of medical trends, track climate changes overtime, and enable credit card default analysis by region,by time, or by individual. For the entertainment indus-

try, the comparable techniques might include re-editingold content, or extracting certain types of scenes or dialog, or re-sizing and re-compressing for new distribution channels that did not exist when the mediaassets were originally created. For example, one studioexecutive described how his company had generatedseveral million dollars in new revenue by extracting certain phrases from old television shows and sellingthese sound bites to consumers as downloadable ringtones for cellular telephones. The Long Tailimplies that studios can maximize their profits byadopting a digital archiving strategy of “save everythingforever,” since everything will have value to someonesomeday. However, some feel that although the Long Tail is very long, it is also very thin, and thereforethe cost, today, of saving everything, may precludeimplementing this strategy.

Where to save – islands of archivingThe number of groups in a major studio capable ofmaking their own digital media is growing. The creative talent and production facilities are becomingdecentralized. Some or all of the content made bythese teams may need to be preserved in digitalarchives. At the same time, the potential distributionchannels served by the studio are proliferating, drivingcreation of more digital formats. One studio reportedthat without a “grand plan” for archiving, they are seeing spontaneous emergence of independent “islands”of digital archives in different business units and func-tional groups. These islands have been developed asstand-alone solutions, and often hold redundant inventory without consistent naming conventions(metadata). There is frequently no inter-operability,even for interchange within the enterprise itself.

According to some, digital archiving operations can add value to assets in terms of potential contentrepurposing far more quickly than analog archives. The question of future accessibility is the basic questionbehind all those decisions. Right now, access overextended periods (100 years or longer) is not guaran-teed in the world of cinema except for YCM separationmasters on film.

6.3 What This Means for the Motion Picture Industry continued

IN

DU

ST

RY

N

EE

DS

A

ND

O

PP

OR

TU

NI

TI

ES

7 Industry Needs and Opportunities


THIS REPORT, THUS FAR, HAS SIMPLY PRESENTED A LARGE COLLECTIONof relevant facts and informed opinions about the creation and preservation of film-based, hybrid and “born-digital” motion picture elements, digital storage technologies,and digital data handling practices occurring both within the motion picture industryand in other industries with similarly large and long-term data storage and preservationneeds. As stated earlier, a primary goal of this work is to provide sufficient informa-tion and grounding so that the motion picture industry’s needs with regard to the tran-sition to a digital infrastructure are clearly defined, ultimately enabling sensible selec-tion and implementation of appropriate technologies and practices that guarantee thelong-term safety of and access to important corporate and cultural assets.

Armed with the perspective of a solid information base, the following sectionsstate, in our view, the most fundamental industry needs regarding the archiving of andaccess to digital motion picture materials. Some of these requirements may appearobvious, but they simply articulate the needs met quite successfully by film technologyfor the entire history of our industry. Although there may not be an equivalent orimproved replacement technology available today, the Science and TechnologyCouncil sees no reason to abandon these needs.

Herein lies the opportunity for the motion picture industry to break from the practice of accepting technologies and methods developed by other industries and busi-ness interests without regard to the most fundamental needs of motion picture productionand preservation. We have the ability to define and communicate our particular needs,leverage the overlapping needs of other industries, and then, perhaps, to have a choice ofsolutions that solve as many problems as the new digital technologies seem to create.

To that end, what follows are the most basic needs of an archive for digital motionpicture materials stated without regard for today’s available solutions:

1. A digital archival system that meets or exceeds the performance characteristics of the traditional film archive

As a starting point, a digital archival system should be at least as capable as thefilm preservation system it replaces in the following respects:

Guaranteed access for at least 100 years: The single characteristic of a digital archivalsystem universally requested by every studio and film archive we spoke with was thataccess to the content stored in the archive should be guaranteed for at least 100 years.Simply put, that is what they have with film, and that is what they want when and iffilm is no longer available.

Immunity from extended periods of neglect and financial hardship: Another characteristic of the film archive is that its contents remain accessible even if it weresubject to, in the words of one studio executive, periods of benign neglect. That is,reductions in staffing or funding would not cause the content to disappear or becomeinaccessible. Although film may slowly degrade if funding shortfalls were to result insuboptimal environmental conditions, restoration would almost always remain an option.

Ability to create duplicate masters to fulfill future (and unknown) distributionneeds: Film archival masters, when properly created and stored, have been of morethan sufficient quality to generate any distribution master, whether for 4K DigitalCinema or handheld portable media players. Any replacement archival technologymust be able to do the same, for both existing and unrealized distribution channels.

Picture and sound quality which meets or exceeds that of original camera negativeand production sound recordings: There is no question that properly created film

A digitalarchival system should be atleast as capable as the filmpreservationsystem itreplaces.


IN

DU

ST

RY

N

EE

DS

A

ND

O

PP

OR

TU

NI

TI

ES

7 Industry Needs and Opportunities continued

archival masters support the generation of distributionmasters with little or no quality loss. The current use of2K and HDTV mastering pipelines, and 2K digital cam-eras for theatrical motion pictures, as well as insufficientattention to image quality during the mastering process,together are generating archival elements that are ofnoticeably lower quality than films created more than 40years ago.

The deployment of 4K Digital Cinema projectionsystems and the introduction of 4K consumer displays16

are clear indications that future display systems willmake greater picture quality demands on the archivalmaster. At a minimum, image quality metrics regard-ing spatial resolution, color gamut and dynamic rangeas defined by SMPTE Digital Cinema standards shouldbe the baseline quality standards, as well as the corre-sponding standards for audio.

No dependence on shifting technology platforms:Film stocks have changed over the years, generally withincreasing quality and stability characteristics (with oneor two notable exceptions). This technology evolutionhas not compromised the accessibility of the film archivalmaster, and therefore a replacement archival technologyshould not subject the archival master to such a risk.17

2. Standardized nomenclature The multi-studio case studies undertaken as part of this report uncovered a problem that has taken morethan 100 years to develop: each studio has a different naming and identification system for the physical and digital objects they create in the manufacture oftheatrical motion pictures. These differences developedfor perfectly logical reasons: each studio’s inventorymanagement system developed organically, along withits internal business systems, so there is no uniformityacross studios. Unfortunately, it is impossible to effec-tively leverage any digital solution with the inefficien-cies this situation creates. Our attempt in the casestudies to simply and accurately quantify the amount of film and digital materials generated during motionpicture production was hampered by the wide variationof inventory management practices. Further refinementof the industry’s needs in this area will be that muchmore difficult without uniform naming practices.

Individual studios will also benefit from such stan-dardization efforts. As part of a recent internal review,one studio identified nine different ways its variousbusiness units referenced a single 60-year-old property[Solomon]. Rationalizing object names not onlyimproves access capability, it also enables strategies that reduce the number of duplicated items taking upvaluable digital storage space. One studio executiveclaimed that “de-duplication” of his studio’s librariesreduced the overall inventory by 30 to 50%.

16 Sharp Electronics 4K LCD shown at CEATEC 2006 and CES 2007.17 The discontinuation of older print stocks has, however, required compensating color timing

and/or correction of older films, since the color characteristics of new print stocks have changed.

CO

NS

EN

SU

S8 Consensus


ALTHOUGH THE PRIMARY INTENT OF THIS DOCUMENT IS TO DEFINEthe problem of digital archiving and shine a light on the important related issues forour industry, there is general agreement among the people interviewed with regard towhat actions to take going forward. The consensus view answers two basic questions:

• What should be done right now?• What should be done over the long term?

Given the conclusion that there is no digital archival master format or process with longevity characteristics equivalent to that of film, the emphasis is on protectingtoday’s assets while work continues on developing suitable long-term solutions.

8.1 To Start

1. Create film separation mastersAs stated earlier, there is virtually unanimous agreement within the industry that filmseparation masters, whether created using three-strip or successive exposure techniques,are a safe and affordable archival master. Some may argue that pure born-digital motionpictures (digitally shot or animated with computerized tools) are degraded when filmgrain, no matter how fine, is added to the images; but the film masters are still wellabove the historical notion of “highest quality,” and are thus far more than capable ofdelivering the quality necessary and expected for all re-purposed distribution needs.

2. Enable the enterprise to develop a rational digital preservation strategy

While there are small groups within each studio that understand the issues presented in this paper, their influence is not exerted until well after the importantdecisions regarding digital asset creation are made. This is too late to ask the questions that must be asked when considering the huge number of choices presentedby digital production and postproduction.

Although every studio ultimately manufactures its products (motion pictures) toidentical delivery specifications (35mm film and Digital Cinema Packages), each organi-zation has distinct internal structures and processes developed over its unique history,influenced by a wide and varying range of non-motion-picture-related business needs.The net effect is that each organization must consider the entirety of its own businessgoals in developing a long-term strategy for archiving and accessing digital materials.

That being said, there are some common elements to be considered:

Accept and understand that preserving digital motion picture materials is fundamentally different than preserving film, and as such, every assumption and practice in motion picture production (including corporate structure) must be looked at from this new perspective. The “save everything” practice used with film is cost-prohibitive with current digital storage technologies, given the huge quantity of data and ongoing preservation expense.

Identify the stakeholders in the enterprise and define their interests, roles and responsibilities with respect to the creation, preservation and access of digital motionpicture assets. We have heard from several studios that the growth of alternate digital distribution channels – television, Internet, mobile, and so on – has fracturedyesterday’s relatively simple asset and inventory management process and corporate

There is nodigitalarchival master format orprocess withlongevitycharacteristicsequivalent to that of film.


CO

NS

EN

SU

S

8.1 To Start continued

structure, and expectations for future fulfillment do not always match up with the realities of current practice. Digital preservation and access must be defined for the enterprise as a whole; e.g.:∑

• What is a long-term asset?• What is considered perishable?• What elements justify the cost of digital preservation?• What are the methodologies under which these decisions are made?

Enable collaboration among the stakeholders to develop a strategy for digital preservation. The following questions raised by using digital technologies cannot be answered by any single department or division:

• What is the value of the content?• Who determines the value of the content?• What content will be archived?• Who determines what content will be archived?• How will the content be archived?• Who determines how the content will be archived?

3. This is an industry problem, and to solve it, the industrymust work together

The founders of the motion picture industry knew early on that their business wasabout selling movies. The mechanisms needed to create their product were simplymeans to that end, and they generally went to great lengths to reduce the costs of production. Collaboration on solving technical problems dates back to 1916 with thecreation of the Society of Motion Picture Engineers (now the Society of MotionPicture and Television Engineers), and 13 years later, at this Academy through itsProducers-Technicians Joint Committee. The standardization of 35mm motion picturefilm, camera aperture and theatrical sound equalization, among other things, was theresult of collaborative efforts by companies that otherwise competed with each other.

The issues of archiving and accessing digital materials are of the same nature: nostudio or filmmaker will make any money from the technological solutions that enablethe long-term preservation of and access to motion picture content. However, unlessand until the issue of long-term access is solved, future revenue streams – and possiblythe art form itself – are highly endangered. The motion picture industry must notnecessarily accept solutions that fall short of what has been used successfully for 100years. The technological solutions are likely to come from outside the industry, but itis vitally important that the industry speak with a common voice on its unique needs.There is also an opportunity to collaborate with other industries that share commonaspects of long-term digital preservation and access, particularly with respect to influencing storage vendors and system solution providers to develop products thatmore closely match our requirements.

4. For the short term, actively protect important digital assetsThere is no denying the reality that over a billion dollars18 has been spent generating digital motion picture assets. Creating YCM separation masters on black-and-whitepolyester film stock protects the final theatrical product, but there may be tremendousvalue remaining in the multiple digital masters generated from a motion picture, andquite possibly in the original digital camera files and tapes. There may be both business

18 Based on the number of digital masters and digitally captured movies to date.

Accept andunderstandthat preservingdigital motion picture materials isfundamentallydifferent than preservingfilm.

CO

NS

EN

SU

S8.1 To Start continued


opportunities and cultural obligations to maintain accessto at least some of these digital objects while the biggerpicture is assessed and long-term strategies are developed.

Design for evolution Unless and until there existsthe digital equivalent of film, i.e., a “store and ignore”preservation medium, organizations will have to managethe realities that hardware and media will become obso-lete, software applications will be upgraded, economicconditions will change, and personnel will come and go.

While we do not at this time accept data migrationas a fait accompli for the industry’s digital preservationfuture, there is no getting around it once one commits tocreating valuable digital assets using commercially avail-able information technology storage products. Whetherthis commitment happens proactively or by default,exclusively using today’s digital technologies makesmigration a necessary, if temporary, strategy to consider.

Design for low risk of technical obsolescenceIt is worth repeating that modern technology productshave finite usable lifetimes, in many cases as short astwo years. However, there are some things that can bedone to mitigate the impact of technology churn:

Standards: If standards exist, use them. File formats,image and sound encoding specifications, and metadataare important work items for the international stan-dards development community, and many of them canbe applied to today’s needs. In the absence of relevantstandards, the industry should organize to create thestandards it needs for digital archiving, much asSMPTE is documenting Digital Cinema distributionspecifications.

Open-source software: There is a large body of open-source software being developed specifically for largedata storage problems. Base software technologies suchas the Storage Resource Broker, the next-generationiRODs distributed storage system, and the LOCKSS(Lots Of Copies Keep Stuff Safe™) program offerinteresting opportunities for minimizing the impact ofchanging vendor strategies and business goals, and pro-prietary single-vendor products.

Lower the risk from threats: economic, technical, human Maintaining digital data for thelong term using today’s technology demands perpetualfunding. Most organizations want to minimize the totaloperating costs of a digital storage system. We want to re-emphasize that the total cost of ownership should bedetermined not only by counting the media costs or theinitial purchase price of the hardware and software, but also

the recurring costs. Furthermore, there are different costfactors to consider when building a digital storage systemand/or outsourcing digital storage services:

In-house Systems: When building a digital storage system, total cost of ownership includes:

• initial hardware, operating system and applicationsoftware costs

• software and hardware maintenance contracts

• replacement costs of hardware, operating systemsand software applications

• external network access costs for distributed systems

• initial and replacement media costs

• personnel costs, including ongoing training

• electrical power and cooling costs

• facilities and real estate costs, taxes and insurance

• increase in costs as digital asset collection grows

• data ingest and access costs

Appropriately sizing the storage system will alsoaffect total cost of ownership. Larger systems tend toreduce the cost per bit, although they require larger initial investments to construct.

Outsourced Systems: When outsourcing a digital storage system, total cost of ownership includes:

• “rental” cost for storage – this can vary widely, depending on the service provider’s level of servicewith respect to the threats described earlier

• data ingest and access costs

• risk mitigation of service provider failure

For both scenarios, another factor that will impacttotal cost of ownership is data duplication. Manyorganizations we spoke with have the common problemof unintended multiple data copies. That is, there aremany redundant copies of motion picture elements,and in the absence of sensible information lifecyclemanagement policies, every bit of data is saved. Thiseasily doubles or triples the amount of data managed(or not managed, as the case may be) by an organiza-tion and this drives storage costs up. “De-duplication”is the practice of eliminating unnecessary redundantfiles, which in turn reduces the amount of data to bestored and the associated cost. That requires anotherempowered decision.


CO

NS

EN

SU

S

8.2 Long Term Initiatives

The motionpicture industryshould organize itself to speak with a common voice on matters ofdigitalarchival technologyand solutions.

THOSE INTERVIEWED FOR THIS REPORT AGREED THAT ACTIONScan be taken to produce better solutions for long-term digital preservation and accessthan we have today. The Science and Technology Council’s goal is to move thesenotions from just being written about to being acted upon.

1. CollaborationsWe stated earlier that the motion picture industry should organize itself to speak with acommon voice on matters of digital archival technology and solutions, thus enabling itto effectively join forces with other industries that have similar needs with respect to digital preservation and access. This is not a problem that can be solved without greatleverage – there needs to be a consortium of end-users, i.e., customers, who can economically scale their demands to make it attractive for vendors to agree to open standards. We point to the audiocassette, CD, 35mm film and for a while, the DVD, as examples of this. Many, many companies were successful in manufacturing, distributing and selling these standardized formats. They did not need proprietary“secret sauce” to be successful in creating and servicing their markets.

There are a number of examples of cross-industry collaboration, the most notableof which, for our purposes, is the National Digital Information InfrastructurePreservation Program (NDIIPP), created by the Library of Congress (discussed earlierin this report). The Library acknowledged that the scope of this problem is simply toolarge for any organization, even the United States government, to tackle on its own.The NDIIPP program currently funds more than 16 external partners working on digitalpreservation research and collections, and the Library is engaged in numerous digitalpreservation-related partnerships with notable institutions including the NationalArchives and Records Administration, the National Science Foundation, and theDigital Library Federation, as well as digital preservation initiatives abroad.

In August 2007 the Academy and the Library of Congress announced theAcademy’s participation in NDIIPP’s Preserving Creative America project, a jointeffort to address the issues of digital preservation as they relate to theatrical motionpictures. Participation in this program will bring increased visibility to the motionpicture industry’s needs, and it is hoped that we will also discover new ideas that willlead to better solutions for the industry. Topic areas of this joint effort include:

• a report on the Digital Dilemma from the perspective of the independent filmmaker and smaller, public film archives

• development of a digital preservation case study system to investigate various digital motion picture archival strategies

• development of requirements and specifications for digital file formats that support long-term digital preservation

• education and research activities related to digital motion picture preservation

This is just one example of the opportunities available to leverage the efforts of severalorganizations and industries toward a common goal.

2. Standards DevelopmentWhile we have heard conflicting advice from other industries on the value of standardswith respect to digital preservation, it is clear that the motion picture industry hasbenefited, and indeed would not exist, without worldwide standards for the interchange of motion picture content. International standards have the added

CO

NS

EN

SU

S8.2 Long Term Initiatives continued


benefit of being automatically reviewed every five years,which provides a built-in mechanism for dealing withthe constant churn of new technology.

We believe that standards are most likely to be successfully implemented and adopted when the usercommunity of those standards takes an active and leadership role in their development. The Society ofMotion Picture and Television Engineers (SMPTE) and the International Organization for Standardization(ISO) are the two accredited standards bodies that publish most of the motion picture industry’s standardsin use today, and both organizations are actively devel-oping standards for Digital Cinema distribution andexhibition. It is interesting and important to note thatthese standards are based on specifications written notby equipment manufacturers and technology providers,but by a consortium of one segment of the user com-munity: the Hollywood studios via the Digital CinemaInitiatives consortium. Much input on the specificationswas taken from another important user group – theexhibitors – as well as from the equipment manufacturers,but the process was driven by a committed and influen-tial user group.

Similar effort must be applied to the ad hoc worldof archiving and access of digital motion picture materi-als. Image file formats, their associated “wrappers,” filenames, metadata, and metadata registries all are of

limited usefulness unless there is industry-wide agree-ment on what they are and how they are to be used.

Compared to motion picture film, motion picturedigital formats are still in their infancy. There is nouniversally accepted standard for all phases in the lifecycle of digital motion pictures assets – production,postproduction, distribution and archiving. The DCIrecommendations and subsequent SMPTE DC28 stan-dards efforts are building consensus around DigitalCinema distribution formats. But the format for theso-called Digital Source Master (DSM), i.e., the digitalequivalent of the cut negative, is not standardized, noris there even agreement on what a DSM is. Digitalcamera acquisition image formats are also not standard-ized. Digital film scanner output formats are not standardized. Technical innovation and market forcestogether are still influencing the evolution of variousdigital formats for Digital Cinema that might one dayhave to be preserved in a digital archive.

Based on the experience to date in television, thedefinition of a Digital Cinema archive master digitalformat will require a detailed evaluation of alternativefile formats, wrappers, image and sound encoding formats, metadata formats and metadata registries. The subject needs a focused effort to build consensusaround one or several digital formats that can be sustained for archival purposes.


CO

NS

EN

SU

S

8.3 Finally...

This is anissue thatrequires top-downexamination, enlighteneddecision-making, and intra-and inter-industry cooperationfor the benefit oftoday's content creators andtomorrow'saudiences.

IN THE CENTURY SINCE CINEMA WAS FIRST INVENTED, MANYdifferent perforation schemes, emulsions and sound track formats evolved and can be found in film archives around the world. Yet today, more than 100 years after itsintroduction, 35mm film is the shining example of a standardized and sustainable format that is widely adopted, globally interoperable, stable and well understood.

The bottom line is that any system proposing to replace photochemical film technology must meet or exceed film’s capabilities. While it is true that end-usersbenefit from new features and cost efficiencies that generally come with new productsand technologies, the economic benefits of technological obsolescence accrue primarilyto the hardware manufacturers and software system developers. In exploring this digital dilemma, it becomes clear that if we allow the historical phenomenon of tech-nological obsolescence to repeat itself, we are tied either to continuously increasingcosts – or worse – the failure to save important assets. This is an issue that requirestop-down examination, enlightened decision-making and intra- and inter-industrycooperation for the benefit of today’s content creators and tomorrow’s audiences.

The Academy was founded to, among other things, represent the viewpoint of the actual creators of motion pictures and facilitate technological progress among the creative leadership of the motion picture industry. It is therefore the proper role ofthe Academy to spotlight this issue by bringing together the resources that producedthis report, and to lead in the actions necessary to solve this dilemma. In addition toinitiating the activities discussed earlier in this report, in the coming months theAcademy will bring together studio decision-makers and technology resources, as wellas other experts, to further define the requirements and issues in the archiving of andaccess to digital motion picture materials. These efforts are a start, but what is alsoneeded is commitment by the primary stakeholders, and objective overview of themanufacturers and system designers, to produce cooperation, standards, and guaran-teed long-term access to created content.

Only then will we have solved this Digital Dilemma for the benefit of all the players.

The place to start is here. The time to start is now.



AP

PE

ND

IX

9 APPENDIX • Case Study Data

The following sections contain summary information from the twocase studies discussed in this report. The subject productions werecaptured either on film or HDCAM SR videotape and mastered at1920 x 1080 pixel count with 10-bit precision per color component.This results in lower total byte counts than 2K/10 bit and 4K/16 bitencoding. However, the number and type of elements identified inthe case studies are believed to be representative of those generated byboth 2K and 4K productions.

The Element Trees were derived from inventory data provided by theparticipating studios. The succeeding Case Study Data Tables containthe actual inventory data (with certain noted assumptions), using theidentifying terms from the Element Trees.

Screen Tests, B Roll,

Deleted Scenes

Editing System Files

35mmDNegative(Archival)

35mmDNegative

35mm Composite

Answer Print

35mm Production

Interpositive

35mmCheck Print

35mm DNegative

Textless

35mm Production

Interpositive

35mm Composite

Answer Print

35mmConformed

Foreign M&E Titles

0.00.20.40.60.81.0

Original CameraNegative

YCM Separation

Masters

Master Files for Textless

DNegative

Digital CinemaDistribution

Master (dcdm)

Master Files

35mm Production

InternegativeDistribution Master

Telecine Dailies

SYMBOL KEY COLOR KEY

ARCHIVAL STORAGE

WORKING LIBRARY

PERISHABLE ELEMENT

DIGITALFILE

VIDEOTAPE35MMFILM

9 APPENDIX • Case Study Data continued


A.1 Generic Element Trees • Picture Element Hierarchy: Film Capture

35mm Production

Interpositive

35mm Production

Internegative

35mmCheck Print

0.00.20.40.60.81.0

35mm DNegative(Archival)

Master Filesfor Final Edited

Picture, Out-Takes and Trims

Master Files

Editing System Files

Master Files for Textless

Digital Negative

35mm DNegative

35mm Composite

Answer Print

Digital Cinema Distribution

Master (DCDM)

YCM Separation

Masters

35mm DNegative

Textless

35mm Production

Interpositive

35mm Conformed Foreign M&E

Titles

Combined Continuity/Master

English SubtitleSpotting List

Screen Tests, B RollDeleted Scenes

Original Production Footage

Cloned Production Footage

Distribution Master


ARCHIVAL STORAGE

WORKING LIBRARY

PERISHABLE ELEMENT

DIGITALFILE

VIDEOTAPE35MMFILM



A.1 Generic Element Trees • Picture Element Hierarchy: Data Capture

0.00.20.40.60.81.0

Original ProductionSound/Field

Sound Recordings

Dialogue StemDiscrete/Stereo

6-Track, 5.1

Orchestra/Scoring Sessions

Music StemDiscrete/Stereo

6-Track, 5.1

Pre-Dubs

Effects StemDiscrete/Stereo

6-Track, 5.1


ARCHIVAL STORAGE

WORKING LIBRARY

Domestic Print Master

5.1

Domestic Stereo LT/RT

Dolby Digital 5.1Lt/Rt

Digital CinemaVersion 5.1

M&E StemDiscrete/Stereo

6-Track, 5.1

M&E LT/RT

Uncompressed 5.1Fully Filled Efx Stem(M&E Minus Music)

Foreign Language Print Master

LT/RT

Foreign Language Print Master

5.1

Foreign DialogueStems

x 11 Languages

LTO2 Copies of All Materials

DVD-R Copies of All Materials

DIGITALFILE

Dolby SR/SRD/SDDS/DTS

OSTN

35MMFILM

A.1 Generic Element Trees • Sound Element Hierarchy: Film or Data Capture




AP

PE

ND

IX


A.2 Case Study Data Tables Table A-1 lists the delivered picture elements identified in the film capture case study plus the mediumof storage, the number of items per element and the estimated file size if digital storage is used.

STORAGE MEDIUM ELEMENT NUMBER FILE SIZEOF ITEMS in TERABYTES

35mm Film 35mm Digitally 178 Cans or Cartons NACreated Negatives35mm Answer Print

35mm Production IP

35mm Production IN

35mm Check Print

35mm Textless DNegative

35mm Textless IP

35mm Textless Answer Print

35mm Foreign Language Main and Ends Negative

35mm YCM Separation Masters

35mm Original Camera Negative

35mm Trims and Outs

LTO2 Data Tape 1920x1080 Master Files1920x1080 Master Files for 15 LTO2 Data Tapes 3 TBTextless DNegative

DVD-R Optical Disk Editing System Files 1 Disk .005 TB

HDCAM SR Videotape Telecine Dailies 486 Tapes 173 TB1

D5 Videotape Distribution Master 9 Tapes .202 TB1

Table A-1 – Delivered Film Capture Picture Elements1 Calculated.

AP

PE

ND

IX



A.2 Case Study Data Tables continued

Table A-2 lists the delivered picture elements identified in the data capture case study plus the medium of storage, the average number of items per element and the estimated file size if stored onmagnetic hard drives or data tapes.

STORAGE MEDIUM ELEMENTNUMBER FILE SIZEOF ITEMS in TERABYTES

35mm Film 35mm Digitally 129 Cans or Cartons NACreated Negatives35mm Answer Print

35mm Production IP

35mm Production IN

35mm Check Print

35mm Textless DNegative

35mm Textless IP

35mm Textless Answer Print

35mm YCM Separation Masters

35mm Conformed Foreign Language Mains and Ends

Magnetic Hard Drives 1920x1080 Master Files 42 Hard Drives 10.7 TB

1920x1080 Master Files for Textless DNegative

1920x1080 Master Files for Outtakes and Trims

Editing Files

HDCAM SR Videotape Original Production Footage 5,347 Tapes 3,257 TB2

Cloned Production Footage/Screen Tests/B-Neg

D5 Videotape Distribution Master 0 Tapes1

DVCAM Videotape Editing System Project Files 728 Tapes 24 TB2

Table A-2 – Delivered Data Capture Picture Elements1 Items not yet delivered to archive.2 Calculated.


AP

PE

ND

IX



Table A-3 lists the delivered sound elements for both film and data capture, the medium of storage,the average number of items per element, and the estimated file size if stored on magnetic hard disks or data tape. The delivered sound element types for both case studies were identical, althoughas expected, the quantities differed between productions because of variances in production and postproduction practices.

STORAGE ELEMENT NUMBER NUMBER FILE SIZE

MEDIUM OF ITEMS OF ITEMS in(Data Capture) (Film Capture) TERABYTES

35mm Film Optical Soundtrack 34 Cans 27 Cans NANegative (OSTN) or Cartons or Cartons

DVD-R 6-Track Dolby Digital 71 DVD-R 371 DVD-R .83 TB Optical Disk Digital Cinema Version (Data Capture)

LT/RT Musefx .42 TB 5.1 Efx Stems (Film Capture)

Foreign Language Dialogue Stems

Foreign Language Print Master (5.1)

Foreign Language Print Master (LT/RT)

LTO2 Data Dolby LT/RT Print Master 13 LTO2 01 .004 TBTape Dolby 5.1 Print Master Data Tapes (Data Capture)

6-Track Dolby Digital

Magnetic Music, Dialogue and 43 Magnetic 3 Magnetic 2.6 TB Hard Drive Effects Stems Hard Drives Hard Drives (Data Capture)

Domestic LT/RT .73 TB Domestic 5.1 Print Master (Film Capture)

5.1 MusefxOrchestra/Scoring Sessions

Table A-3 – Delivered Sound Elements 1 Items not yet delivered to archive.

AP

PE

ND

IX




Table A-4 lists the elements, the number of items and the storage category assignment for pictureand sound elements from the film capture production. This breakdown may vary depending on thestudio because of differing practices. Items in bold exist only in the film capture production and notin the data capture production.

As stated previously, “archival” is defined as storage of the master elements from which all down-stream distribution materials can be created over a 100-year timeframe, and “working library” storageis a broad term for elements that are generally kept on hand for distribution purposes.

STORAGE CATGEGORY PICTURE ELEMENT SOUND ELEMENT

Archival

Mixed Archival/Working Library

Working Library

35mm YCM Separation MastersLTO – 1920x1080 Master Filesfor Digital Negative35mm Digital Negative35mm Composite Answer Print(from Digital Negative)35mm Digital Negative Textless

35mm Original CameraNegative35mm Production InternegativeHDCAM SR – Screen Tests, B-Roll, Deleted ScenesHDCAM SR – Dailies

DVD-R – Editing System FilesLTO – 1920x1080 Master Filesfor Textless Digital NegativeD5 – Distribution Master35mm Production Interpositive35mm Check Print35mm Production IP, Textless35mm Answer Print, Textless35mm Foreign Language Mains and Ends35mm Trims and OutsDVD-R – CombinedContinuity/English SubtitleSpotting List

LTO or DVD-RCopies of all Working LibrarySound Materials

NA

Original Production SoundPre-DubsOrchestra/Scoring SessionsDialogue StemsEffects StemsMusic StemsDolby Stereo LT/RTDolby SR/SRD/SDDS/DTS OSTNDomestic Print MasterDolby Digital LT/RTMusefx Stem Discrete 6-track, 5.1Musefx LT/RT5.1 Fully Filled Efx StemForeign Language Dialogue StemsForeign Language Print Master 5.1Foreign Language Print Master LT/RT

Table A-4 – Storage Categories for Picture and Sound Elements from a Film Capture Production


AP

PE

ND

IX



Table A-5 lists the elements, the number of items and the storage category assignment for pictureand sound elements from the data capture production. Again, this may vary depending on the studiobecause of varying practices. Items in bold exist only in the data capture production.

STORAGE CATGEGORY PICTURE ELEMENT SOUND ELEMENT

Archival

Mixed Archival/WorkingLibrary

Working Library

35mm YCM Separation Masters Hard Drives – 1920x1080Master Files for Digital Negative35mm Digital Negative35mm Composite Answer Print(from Digital Negative)35mm DNegative Textless

Hard Drives – 1920x1080Master Files for Final EditedPicture, Outtakes and TrimsHDCAM SR – Screen Tests, B-Roll, Deleted Scenes

HDCAM SR - OriginalProduction FootageHDCAM SR– Cloned Production FootageDVD-R – Editing System FilesLTO – Master Files for Textless Digital NegativeHDCAM SR – DistributionMaster35mm Production Interpositive35mm Production Internegative35mm Check Print35mm Production IP, Textless35mm Foreign Language Mains and Ends

LTO or DVD-RCopies of all Working LibrarySound Materials

NA

Original Production SoundPre-DubsOrchestra/Scoring SessionsDialogue StemsEffects StemsMusic StemsDolby Stereo LT/RTDolby SR/SRD/SDDS/DTS OSTNDomestic Print MasterDolby Digital LT/RTMusefx Stem Discrete 6-track, 5.1Musefx LT/RT5.1 Fully Filled Efx StemForeign Language Dialogue StemsForeign Language Print Master 5.1Foreign Language Print MasterLT/RT

Table A-5 – Storage Categories for Picture and Sound Elements from a Data Capture Production

AP

PE

ND

IX




Baseline Storage Costs

As stated previously, the baseline storage costs used for this study are:

• $4.80 per physical item per year for archival storage

• $1.80 per physical item per year for working library

• $500 per terabyte per year for near-line data tape storage (single copy)

• $1,500 per terabyte per year for online magnetic hard drive storage (single copy)

Initial inspection and access costs are not included in the baseline film storage costs, nor are access oringest costs included in the baseline digital storage costs because reliable information for the latter isnot available. Nonetheless, these costs should be considered when considering the type and quantityof assets being stored.


AP

PE

ND

IX



Table A-6 lists the estimated annual cost of storing the delivered picture elements from the film capture production. This includes digital elements that are created during postproduction and storedon LTO2 data tape, HDCAM SR (or equivalent) videotape, and DVD-R optical disk. Items in bold are stored in archival conditions and are so noted.

Today, the practice in Hollywood is to store digital media as physical items in either archival orworking library conditions. Given the special handling requirements of digital data, and the associated costs, the following table calculates the estimated cost of storing the digital elements separately as data, on data tape, in a fully managed environment consistent with the archival intent.

Table A-6 – Estimated Annual Cost of Element Storage – Film Capture

35mm Film

LTO2 Data Tape

DVD-ROptical Disk

HDCAM SRVideotape

D5 Videotape

35mm Digital Negative35mm Answer Print35mm Production IP35mm Production IN35mm Check Print35mm Textless DigitalNegative35mm Textless IP35mm Textless Answer Print35mm Foreign Mains andEnds Negative 35mm YCM SeparationMasters35mm Original CameraNegative35mm Trims and Outs

1920x1080 Master Files forDigital Negative1920x1080 Master Files forTextless Digital Negative

Editing Files

Telecine Dailies

Distribution Master

$1,5061

(Archival)$290 (Working Library)

$72 (Archival)

$2 (Working Library)

$2,333 (Archival)


NA

$1,465 (Archival)

$2 (Archival or Working Library)

$86,498 (Archival)


STORAGE MEDIUM ELEMENT

ANNUAL STORAGECOST OF DELIVERED

ITEMS

ANNUAL FULLYMANAGED

STORAGE COSTIF STORED ON

DATA TAPE

1 Includes amortized cost of YCM separation master manufacture, which is $800 per year.

AP

PE

ND

IX




Table A-7 lists the estimated annual cost of storing the delivered picture elements from the data capture production. This includes both born-digital elements and digital elements created duringpostproduction, and stored on magnetic hard drive or HDCAM SR (or equivalent) videotape. Itemsin bold are stored in archival conditions and are so noted.

The estimated cost of storing a single copy of born-digital materials on data tape is also calculated torepresent the use of uncompressed digital data recorders now in use.

ELEMENT

Table A-7 - Estimated Annual Cost of Element Storage – Data Capture

35mm Film

MagneticHard Drives

HDCAM SRVideotape

DVCAM

35mm Digital Negative35mm Answer Print35mm Production IP35mm Production IN35mm Check Print35mm Textless Negative35mm Textless IP35mm Textless AnswerPrint35mm YCM SeparationMasters

1920x1080 Master Files for Digital NegativeComplete 1920x1080Master Files for TextlessComplete 1920x1080Master Files, Outtakes andTrims

Original Production FootageCloned Production Footage/Screen Tests/B-Neg

Editing System Project Files

$1,1021 (Archival)$124(Working Library)

$64 (Archival)

$1,170 (Working Library)


NA

$5,127 (Archival)

$1,629,128(Working Library)

$11,245 (Working Library)


ANNUAL STORAGECOST OF DELIVERED

ITEMS

ANNUAL FULLYMANAGED

STORAGE COSTIF STORED ON

DATA TAPE

1 Includes amortized cost of YCM separation master manufacture, which is $800 per year.


AP

PE

ND

IX



As stated earlier, the trend in the audio domain, where all delivered elements are born digital, is tocopy all master audio files to DVD-R and LTO3 and geographically separate these materials for protection. The master files remain in the working library on magnetic hard drives for instant access.This approach is believed to be the most comprehensive attempt to create an archival process around motion picture sound elements, provided that the process includes a data integrity check andmigration plan that outlives economic and labor factors.

Table A-8 lists the estimated annual cost of storing the delivered sound elements from the case studyproductions. The estimated annual cost of storing the sound elements on magnetic hard drives isalso included to reflect current practice at certain studios.

ELEMENT

Table A-8 – Estimated Annual Cost of Sound Element Storage

35mm Film

DVD-ROptical Disk

LTO2 Data Tape

Magnetic Hard Drives

OSTN

Multi-ChannelMaster Stems5.1 DomesticPrintmasterDomestic LT/RT6-Track Dolby DigitalDigital Cinema Version5.1 MusefxLT/RT MusefxProduction SoundPre-Dubs5.1 Efx StemForeign DialogueStemsForeign LanguagePrint Master (5.1)Foreign LanguagePrint Master (LT/RT)Copies for GeographicSeparation

Dolby LT/RT, 5.1,6 Track Print Masters

Music, Dialogue andEffects StemsDomestic LT/RT, 5.1 Print MasterMusefxOrchestra/Scoring Sessions

$61 (Data Capture:Working Library)$49 (Film Capture:Working Library

$144 (Data Capture:Archival)$668 (Film Capture:Working Library)

$4 (Data Capture:Working Library)

$79 (Data Capture:Working Library)$5 (Film Capture:Working Library)

NA

$414 (Data Capture:Archival)$212 (Film Capture:Working Library)


$1,222 (Data Capture:Working Library)$366 (Film Capture:Working Library)

NA

$1,242(Data Capture:Archival)$635(Film Capture:Working Library)


$3,667(Data Capture:Working Library)$1,099 (Film Capture:Working Library)


ANNUAL STORAGE COST OF DELIVERED

ITEMS

ANNUALFULLY MANAGEDSTORAGE COSTIF STORED ON

DATA TAPE

ANNUAL FULLY MANAGEDSTORAGE COSTIF STORED ONHARD DRIVE

RE

FE

RE

NC

ES

10 References


Anderson, Chris. “The Long Tail.” Wired 12.10 Oct. 2004. CondeNet Inc. 17 Aug. 2007 http://www.wired.com/wired/archive/12.10/tail.html.

Baker, Mary, et al. “A Fresh Look at the Reliability of Long-Term Digital Storage.” Proceedings of the 2006 Eurosys Conference, April 18-21, 2006: Leuven, Belgium. ACM Press, New York, NY, 221-234. 22 Jan. 2007 http://www.lockss.org/locksswiki/files/3/30/Eurosys2006.pdf.

DCinema Today. “Digital Installations.” Sept. 2007. WHM Cinema Consulting, LLC. 21 Sept. 2007. http://www.dcinematoday.com/dcdb/DCinemas.aspx?app=1&cnty=49&dci=1.

Digital Imaging and Communications in Medicine. “Part 1: Introduction and Overview.” National Electrical Manufacturers Association PS 3.1-2007: 1-21. Rosslyn, VA. 15 Aug. 2007 http://medical.nema.org/dicom/2007/.

Galloway, Stephen. “Studios Are Hunting the Next Big Property.” The Hollywood Reporter 10 July 2007. Nielsen Business Media, Inc. 16 July 2007 http://www.hollywoodreporter.com/hr/content_display/news/e3if727c623f03c782b8ad564866c828796.

Global Storage Technologies Inc. “GST’s Annual State of Tape Report for 2004.” GST Research and Engineering Group 2004: 1-15. Lake Forest, CA. 7 Sept. 2007 http://www.gstinc.com/white/report-2004.pdf.

Herman, Robert. Telephone interview with Laurin Herr. 1 Feb. 2007. Oakdale, MN, USA.

Hogan, Hank. “Film Fades Out.” Photonics Spectra Mar. 2006. Laurin Publishing Co., Inc. 1 Oct. 2007. http://www.photonics.com/content/spectra/2006/March/business/68479.aspx.

Holmes Jr., David R., Merrill A. Wondrow and Joel E. Gray. “Isn’t it Time to Abandon Cine Film?” Catheterization Cardiovascular Diagnosis 20.1 1990: 1-4. PubMed Abstract. 22 Aug. 2007 http://www.ncbi.nlm.nih.gov/sites/entrez?db=PubMed.

Hurwitz, Matt. “Crime Scenes and Compression Schemes: The File-Based Workflow for David Fincher’s Zodiac.” Videography 15 Mar. 2007. 19 July 2007http://www.videography.com/articles/article_14970.shtml.

Karagosian, Michael. “Multichannel Film Today.” MPKE Consulting LLC Sept. 2003. 4 Sept. 2007 http://www.mkpe.com/publications/digital_cinema/misc/multichannel.php.

Kienzle, Claudia. “Breaking News . . . Disc at 11: The Migration to Tapeless News Production.” TV Technology 11 July 2007: 1 & 12.

Kienzle, Claudia. “Taking the Tapeless Archive Challenge.” TV Technology 11 July 2007: 12.

Kirsner, Scott. “Digital Cinematography: The Big Pixel.” The Hollywood Reporter 13 June 2006. Nielsen Business Media, Inc. 19 July 2007http://www.hollywoodreporter.com/hr/search/article_display.jsp?vnu_content_id=1002688109.

Kushner, David. “One Giant Screwup for Mankind.” Wired 15.01 Jan. 2007: 1-3. CondeNet Inc. 15 July 2007http://www.wired.com/wired/archive/15.01/nasa.html?pg=1&topic=nasa&topic_set=.

Lorie, Raymond. “The UVC: A Method for Preserving Digital Documents: Proof of Concept.” IBM/KB Long-Term Preservation Study Report Series Number 4. International Business Machines Corporation and Koninklijke Bibliotheek. Dec. 2002: 1-47. Amsterdam, Netherlands. 18 Aug. 2007 http://www.kb.nl/hrd/dd/dd_onderzoek/reports/4-uvc.pdf.


RE

FE

RE

NC

ES

10 References continued

Lyman, Peter and Hal R. Varian. “How Much Information? 2003.” School of Information Management and Systems 30 Oct. 2003: 1-112. University of California, Berkeley. 3 Aug. 2007http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/printable_report.pdf.

Mellor, Chris. “Super-tape Market Edges Toward Monopoly.” Techworld.com. 30 Jun. 2005. International Data Group. 17 Sept. 2007 http://www.techworld.com/storage/news/index.cfm?newsid=3951.

Moore, Richard L. et al. “Disk and Tape Storage Cost Models.” San Diego Supercomputer Center, University of California, San Diego, CA: La Jolla, 2007.

Motion Picture Association of America. “2006 U.S. Theatrical Market Statistics: Worldwide Market Research & Analysis.” mpaa.org 2006: 1-24. 24 Aug. 2007 http://www.mpaa.org/2006-US-Theatrical-Market-Statistics-Report.pdf.

National Research Council of the National Academies. Building an Electronic Records Archive at the National Archives and Records Administration: Recommendations for a Long-Term Strategy. Ed. Robert F. Sproull and Jon Eisenberg. Washington, D.C.: The National Academies Press, 2005.

Overfelt, Maggie. “Saving the Cineplex.” CNNMoney.com 19 July 2007. DCinema Today. 22 Aug. 2007 http://www.dcinematoday.com/dc/extURLs.aspx?ID=290.

Palm, Jonas. “The Digital Black Hole.” Swedish National Archives 2006: 1-14. Stockholm, Sweden. 19 July 2007 http://www.tape-online.net/docs/Palm_Black_Hole.pdf.

Pinheiro, Eduardo, Wolf-Dietrich Weber and Luiz André Barroso. “Failure Trends in a Large Disk Drive Population.” Paper presented at the 5th USENIX Conference on File and Storage Technologies – Paper, February 12-13, 2007: 1-13 San Jose, California. 8 Aug. 2007 http://209.85.163.132/papers/disk_failures.pdf.

Rosenthal, David S., et al. “Requirements for Digital Preservation Systems.” D-Lib Magazine 11, (11) 2005. 19 June 2007 http://www.dlib.org/dlib/november05/rosenthal/11rosenthal.html.

Scherzer, Barbara. “Cookson Puts Focus on Classics.” Variety 15 Apr. 2007. Reed Business Information. 20 Aug. 2007 http://www.variety.com/index.asp?layout=print_story&articleid=VR1117963134&categoryid=2222.

Schroeder, Bianca and Garth A. Gibson. “Disk Failures in the Real World: What does an MTTF of 1,000,000 Hours Mean to You?” Paper presented at the 5th USENIX Conference on File and Storage Technologies – Paper, February 12-13, 2007: San Jose, California. 26 Mar. 2007http://www.usenix.org/events/fast07/tech/schroeder/schroeder_html/index.html.

Science Applications International Corporation. “Architecture and Technology Program: Offline Archive Media Trade Study.” United States Geological Survey. 2006: 1-34. USA. Aug. 2007 http://wgiss.ceos.org/archive/archive.pdf/FY06%20Media%20Trade%20Study.pdf.

Solomon, Madi. Presentation. Digital Asset Symposium. Association of Moving Image Archivists. Pickford Center for Motion Picture Study, Linwood Dunn Theater, Hollywood, CA, 11 May 2007.

Taylor, Jeff and Tom Regal. Presentation. Digital Asset Symposium. Association of Moving Image Archivists. Pickford Center for Motion Picture Study, Linwood Dunn Theater, Hollywood, CA, 11 May 2007.

RE

FE

RE

NC

ES

10 References continued


United States. Library of Congress. Digital Preservation. What the Library is Doing. The Library of Congress. 29 Aug. 2007 http://www.digitalpreservation.gov/library.

United States. Library of Congress. National Audio-Visual Conservation Center – Culpeper, Virginia: Computer Server and Storage Infrastructure RFP LOC-NAVCC-06-01. Washington, D.C.: Library of Congress, 2006.

United States. Library of Congress. National Film Preservation Board. National Film Preservation Plan: An Implementation Strategy. 22 Jan. 2004. 22 Aug. 2007 http://www.loc.gov/film/implemen.html.

United States. Library of Congress. National Film Preservation Board. Television/Video Preservation Study: Volume 1: Report. Oct. 1997. 28 Aug. 2007 http://www.loc.gov/film/tvstudy.html.

United States. National Science Board. National Science Foundation Long-lived Data Collections Enabling Research and Education in the 21st Century NSB-05-40 2005: 1-92. 3 Aug. 2007 http://nsf.gov/pubs/2005/nsb0540/start.html.

Van Bogart, John W. C. “Magnetic Tape Storage and Handling: A Guide for Libraries and Archives.” Washington, DC: Commission on Preservation and Access; St. Paul, MN: National Media Laboratory, June 1995: 1-48. 17 Aug. 2007 http://www.imation.com/government/nml/pdfs/AP_NMLdoc_magtape_S_H.pdf.

The X Lab. “Optical Media Longevity.” 2007. 18 Sept. 2007 http://www.thexlab.com/faqs/opticalmedialongevity.html.

Bibliography

Advanced Distributed Learning. 2007. Office of the Under Secretary of Defense for Personnel and Readiness. 28 Aug. 2007 http://www.adlnet.gov.

Consultative Committee for Space Data Systems. “Reference Model for an Open Archival Information System (OAIS).” CCSDS 650.-B-1. Blue Book, Issue 1. Jan. 2002: 1-148. Washington, D.C. 8 Aug. 2007 http://public.ccsds.org/publications/archive/650x0b1.pdf.

Film Restoration & Conservation Strategies. 2004. European Film Heritage on the Threshold Of The Digital Era, The First Project’s Final Report, Conclusions – Guidelines – Recommendations. Brussels: Gabrielle Claes, Royal Film Archive.

i Rule Oriented Data Systems. Aug. 2007. San Diego Supercomputer Center. 27 July 2007 http://irods.sdsc.edu/index.php/Main_Page.

Linear Tape-Open Technology. Ultrium Roadmap. 2007. Hewlett-Packard, International Business Machines Corporation, and Quantum. 19 Sept. 2007http://www.lto-technology.com/technology/uroad.php?section=0&subsec=uroad.

Lots of Copies Keep Stuff Safe. Sept. 2007. Stanford University. 27 July 2007http://www.lockss.org/lockss/Home.

Storage Resource Broker. June 2007. San Diego Supercomputer Center. 27 July 2007 http://www.sdsc.edu/srb/index.php/Main_Page.


AC

KN

OW

LE

DG

ME

NT

S

11 Acknowledgments

Science and Technology Council

Ray Feeney, Co-chairBill Taylor, Co-chairAndy Maltz, Director

Digital Motion Picture ArchivalProject Committee

Phil Feiner, ChairMilt Shefter, Report Project LeadSteven Anastasi*Craig BarronElizabeth Cohen*Grover CrispClay Davis*Carl FleischhauerRob Hummel*Brad Hunt*Tim KittlesonPhil LelyveldEdrolfo LeonesGregory LukowGary MorseMichael O’BrienMichael PogorzelskiEddie RichmondGarrett Smith*Madi SolomonCharles Swartz* (in memoriam)

Laurin Herr, Consultant & Lead Interviewer

Von Johnson, Case Studies Consultant

*Report Project Committee Member

The Council would also like to thank all of the individuals, studios and other organizationsthat participated in the creation of this report:

David ArcturMary BakerArt BeckmanPeter BoerKen BoettcherRichard BuchananLaura CampbellDave CavenaRobert CecilChris CooksonBrian DavisJeff DavisJohn FaundeenMichael FriendEdward GroganJim HannafinWade HannibalMargaret HedstromRobert HermanScott KellyGlenn KennelMark KimballBob LambertJim LindnerStephen LongHoward LukkTad MarburgBrian McKayRichard MooreLiz MurphyBob O’NeilMark OstlundPeter OttoRick PictonJerry PierceBrian SaundersGreg Servos

Tim SmithGreg ThagardKen ThibodeauRick UtleyRudy ValdezChris WoodBob Yacenda

Amazon.com, Inc.Ascent Media GroupThe Cleveland Clinic FoundationEastman KodakHess CorporationHP LabsImation CorporationInPhase TechnologiesLaser Pacific Media CorporationLibrary of CongressMicrosoft CorporationNational Archives and Records

AdministrationNational Security AgencyNBC/Universal StudiosOffice of the Director of

National IntelligenceOpen Geospatial ConsortiumOvation Data Services, Inc.Paramount PicturesPro-tek Media

Preservation ServicesQuantumSan Diego Supercomputing

CenterSony Pictures EntertainmentSun Microsystems, Inc.20th Century Fox Film

CorporationUnited States Geological Survey,

Center for Earth Resource Observation and Science

University of MichiganThe Walt Disney CompanyWarner Bros.

This report would not have been possible without the support of the members of the Science andTechnology Council’s Digital Motion Picture Archival Project Committee, the consultants who performed the research and interviews, and the many experts from the motion picture industry andother sectors who generously provided information and insight on the Digital Dilemma.

This report, in its various draft forms, was reviewed by several of the individuals listed below to assist theCouncil with delivering a final version that meets Academy standards for overall quality and accuracy.The reviewers’ comments and identities will remain confidential to protect the peer review process. The reviewers have not seen the final version of this report, nor do they necessarily support the report’sconclusions and recommendations. The Academy of Motion Picture Arts and Sciences is whollyresponsible for the contents of the final report.

ABOUT THE SCIENCE AND TECHNOLOGY COUNCIL

The mission of the Science and Technology Council of the Academy of Motion Picture Arts and Sciences is:

• To advance the science of motion pictures and foster cooperation for technological progress in support of the art

• To sponsor publications and foster educational activities that facilitate understanding of historical and new developments both within the industry and for the wider public audience

• To preserve the history of the science and technology of motion pictures

• To provide a forum and common meeting ground for the exchange of information and to promote cooperation among divergent technological interests, with the objective of increasing the quality of the theatrical motion picture experience

For more information on the Science and Technology Council, visit http://www.oscars.org/council.

THEDIGITALDILEM

MA:S

TRA

TEG

IC IS

SUES

IN A

RCH

IVIN

G A

ND

AC

CES

SIN

G D

IGIT

AL

MO

TIO

N P

ICTU

RE M

ATE

RIA

LS