Upload
dayna-gardner
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
““Storage is Cheap” and Other LiesStorage is Cheap” and Other Lies
Lance Stuchell, University of Michigan LibraryCurating and Managing Research Data for Re-UseICPSR, July 31, 2013
“Hard Drives” by Michael Muni CC-BY-NC-ND
Overview
• Instability of storage hardware costs • Check out David Rosenthal’s blog at
http://blog.dshr.org/
• Costs of storing digitized moving images
• Effects of storage costs on digital preservation, formats, etc.
“Storage is Cheap”
• Perception is based on “Kryder's Law”• A 2005 Scientific American article about Mark
Kryder, Seagate's Senior VP of Research • Magnetic disk density increases quickly• Disk density closely tied to pricing • 30-year history of disk prices dropping about
40% per year
• Disk costs were affordable & predictable• 30% of total storage costs
The Party's Over
• Mid 2011: Kryder’s Law slows• Latest projections ~ 20% density growth
• Late 2011: Flooding in Taiwan causes shortfall of 70 million disk drives• Prices remain over 50% higher• Not expected to return to pre-flood levels
until 2014
• Changes in technology David S. H. Rosenthal, “Storage Will Be A Lot Less Free Than It Used To Be,” 2012.http://blog.dshr.org/2012/10/storage-will-be-lot-less-free-than-it.html
“Optimistically, for the rest of this decade the rapid decrease in cost per bit of storage that
has been a constant of the last three decades will be much slower; it might even stop.”
David S. H. Rosenthal, et al., “The Economics of Long-Term Digital Storage” 2012.http://www.lockss.org/locksswp/wp-content/uploads/2012/09/unesco2012.pdf
A/V Digitization at MLibrary
Digitization of Special Collections material
Audio:• Uncompressed BWF master files• 1 original audio object ≈ 5 GB storage
Moving Image:• Uncompressed preservation master• Compressed production master• 1 original video object ≈ 40.4 GB Storage
A/V Storage Estimates
A/V Storage EstimatesTB/Year
Initial Costs Estimates
Value Storage: $250/TB per year
Initial Costs Estimates
Value Storage:Tape Backups:
$250/TB per year$1,825/TB per year
Initial Costs Estimates
Value Storage:Tape Backups:
$250/TB per year$1,825/TB per year
2018: $245,3252023: $485,200
2013: $32,450
Reaction
Current Costs Estimates
Value Storage:Tape Backups
(MLib):
$250/TB per yearEquipment costsCheaper most years
Current Cost Estimates
Current Cost Estimates
Current Cost Estimates
Total 11 Year Storage Cost
$569,368
Current Cost Estimates
Current Cost Estimates
Looking Ahead
• Assuming steady pricing• Instability in per-bit storage
• Will not fall under 20% growth until 2018
• ITS storage costs may not fall at same rate
• Long-term costs of cloud are not known• Amazon costs haven't decreased at HD rate1
• Tape storage for preservation copies?
• Economies of scale• Digital Preservation Network
1Rebecca Pool, “Is cloud storage the answer to preservation?” Research Information, 2/14/2013
Ramifications
• Preservation• Some moving image material is not
retained in uncompressed formats• Still image formats are balance of
preservation and size
• Appraisal and re-appraisal • What are we keeping and for how long?
• Costs of starting up
Storage Takeaways
• HD costs are no longer predictable • Budgets often neglect storage
• Ignoring significant and ongoing costs
• Storage costs are community wide problem• Question of scale• Best practice may not be possible
• Backups can cost more than primary storage
• Archival storage ain’t cheap!
Thanks!!