Embed Size (px)
Citation preview
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
AWS Big DataJon Einkauf
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Agenda• Brief overview of AWS Big Data services• Demo (Query logs in S3 using Amazon EMR)• Q&A
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Technologies and techniques for working productively with data, at any scale.
Big Data
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Big data and AWS
Big data Cloud computing
Potentially massive datasets
Virtually unlimited capacity
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Big data and AWS
Big data Cloud computing
Iterative, experimental style of data manipulation and analysis
Iterative, experimental style of infrastructure deployment/usage
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Big data and AWS
Big data Cloud computing
Frequently not steady-state workload; peaks and valleys
At its most efficient with highly variable workloads
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Big data and AWS
Big data Cloud computing
“Time to results” is critical; shared resources are a bottleneck
Parallel compute projects allow each workgroup to have more autonomy, get faster results
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Ease of useLower costs
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Only pay for what you use
No capital investment
Pay as you go
Lower costs
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Programmable
Integrate with existing tools
Low admin
Easy to configure
Ease of use
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Use the right tools
Amazon S3
Amazon Kinesis
Amazon DynamoDB
Amazon Redshift
Amazon Elastic
MapReduce
AWS Data Pipeline
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Amazon S3
• High scalable object store• 99.999999999% durability• Encryption• Data lifecycle management
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Amazon Kinesis
• Real-time processing• High throughput• Elastic• Integrates with EMR, S3,
Redshift, DynamoDB
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Amazon DynamoDB
• NoSQL database• Seamless scalability• Low admin• Single digit millisecond latency
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Amazon Redshift
• Relational data warehouse• Massively parallel• Petabyte scale• Fully Managed• Low cost ($1K/TB/Year with
3 year Reservation)
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Amazon Elastic
MapReduce (EMR)
• Managed Hadoop clusters• MapReduce, Hive, Pig,
Impala, HBase, Spark, Accumulo, etc.
• Integrates with S3, DynamoDB, Redshift, Data Pipeline, Kinesis
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
AWS Data Pipeline
• Data-driven workflows• Integrates with EMR, EC2,
S3, Redshift, DynamoDB, SNS
• Process and move data between AWS and your own data center
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Log Analysis Example
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Demo
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Big Data on AWS
Brand new course on Big Data
aws.amazon.com/training/course-descriptions/bigdata
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
AWS Big Data Test Drives
APN Partner-provided labs
aws.amazon.com/testdrive/bigdata
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
https://aws.amazon.com/training
AWS Training & Events
Webinars, Bootcamps, and Self-Paced Labs
aws.amazon.com/events
AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
Thank [email protected]
