(SOV203) Understanding AWS Storage Options | AWS re:Invent 2014

  • View

  • Download

Embed Size (px)


With AWS, you can choose the right storage service for the right use case. This session shows the range of AWS choices-from object storage to block storage-that is available to you. We include specifics about real-world deployments from customers who are using Amazon S3, Amazon EBS, Amazon Glacier, and AWS Storage Gateway.

Text of (SOV203) Understanding AWS Storage Options | AWS re:Invent 2014

  • 1. November 12, 2014 | Las Vegas, NV Guy Farber, AWS Storage Business Development

2. AWS storage options Customer presentation: Ooyala 3. We are constantly producing more data 4. From all types of industries 5. AMAZON S3 Simple Storage Service 6. Of Unique Customer Objects 7. peak transactions per second 8. durability 9. 99.99% saves ~20% 10. Amazon S3 11. Amazon S3 12. 1 PB raw storage 800 TB usable storage 600 TB allocated storage 400 TB application data S3 pay only for what you use! Amazon S3 13. Amazon S3 Amazon S3 Amazon S3 14. Amazon S3 Amazon CloudFront 15. Low-cost archiving service 16. per GB / month 17. per TB / year 18. durability 19. data retrieval 20. vs. 21. Amazon Glacier 22. Policy-based archiving service 23. Amazon S3 24. Amazon Glacier Worlds leading social sound platform Audio files must be transcoded and stored in multiple formats S3 Amazon Glacier 25. S3 Amazon Glacier 26. Elastic Block Storage 27. 10GB 1TB 28. Price Performance Latency Business impact EBS Magnetic General Purpose Provisioned IOPS Use Cases Infrequent data access Boot volumes Small to Med DBs Dev and Test I/O intensive Relational DBs NoSQL DBs Storage Media Magnetic disk-backed SSD-backed SSD-backed Performance 100 IOPS on average. Ability to burst to hundreds of IOPS Burst to 3,000 IOPS maximum, with a base performance of 3 IOPS/GiB Consistently performs at provisioned level, 4000 IOPS maximum Price $.05/GB - Month $.05/million I/O $.10/GB Month $.125/GB - Month $.065/provisioned IOPS 29. Amazon EBS Amazon EC2 30. AWS Storage Gateway Your on-ramp to AWS cloud storage: Backup into Amazon S3 DR on Amazon EC2 Archive into Amazon Glacier For a deeper dive, join: BAC202 - Introducing AWS Solutions for Backup and Archiving 31. Performance-oriented Cost-oriented 32. CLOUD STORAGE AT OOYALA Ilan Rabinovitch Manager, Infrastructure and Site Reliability Engineering 33. HISTORY OF AWS AT OOYALA Ooyala born in the cloud in 2007 Winner of the first AWS Startup Challenge in 2007 Our mission is to help our customers bring personalized video to every screen everywhere Today we use a number of AWS services including: Amazon S3 Amazon EC2 + Amazon VPC Amazon CloudFront AWS Direct Connect 34. Storage use cases 35. Log storage with Amazon S3 and Amazon Glacier As a data-focused company, we log and analyze EVERYTHING Needed a scalable and cost-effective way to transport, store, and access billions of log files Enter Amazon S3 and Amazon Glacier Store application logs and telemetry data in S3 Download and process in Hadoop and Spark Archive older data to Glacier to reduce storage costs 36. Log storage with Amazon S3 and Amazon Glacier Infinitely scalable storage with S3 Automatically archive older data to Glacier via life cycle polices Low latency connectivity to on- premises facilities via Direct Connect Benefits Tips Know your access patterns: Large batch restorations can be time intensive and costly Plan ahead! S3 Import / Export requires you to thaw from Glacier first Secure your data with custom IAM keys and instance roles 37. WALLE: Safe deletion in S3 Ooyala stores and transcodes customer videos in S3 as origin for CDNs Overtime needed to delete over 200 million stale files safely Enter WALLE: Internal API for asset deletion Spawns Spot Instances based on workload to issue deletion requests S3 copy to temporary Trash bucket with lifecycle policies 38. WALLEWALLE: Safe deletion in S3 Tips Spring Cleaning Can Generate Real Savings Automate your data retention and deletion with lifecycle policies Safety First Use S3 versioning or temporary storage to protect against mistakenly overwriting or deleting data Scale Cost-Effectively Use Spot Instances and Auto Scaling where possible to keep costs in check 39. Client-side web applications Ooyala develops web applications, which our clients use to manage their content libraries Many applications have migrated to client-side JavaScript to allow for a richer and more performant user experience Needed a simple way to deploy and scale these client-side applications Enter S3 websites and CloudFront Upload static content such as HTML + JS to S3 Accelerate and cache content via Amazon CloudFront Use AJAX to interact with server side APIs and data 40. Client-side web applications Easy and repeatable deployments Scales easily and securely; no web servers to manage Low-cost, highly performant delivery via CloudFront Benefits Tips Reuse S3 buckets: AWS allows up to 100 S3 buckets per account; use CloudFront behaviors and IAM to host multiple domains on a single S3 bucket Cache-control headers Dont forget to set CacheControl headers on your objects to ensure caching benefits Secure delivery: Use CloudFront to provide SSL delivery with custom certificates on your domain 41. Learn more about AWS at Ooyala: engineering.ooyala.com Join our team and build on the cloud: ooyala.com/careers THANK YOU 42. http://bit.ly/awsevals