Upload
eric-tendian
View
981
Download
1
Tags:
Embed Size (px)
DESCRIPTION
In this talk I gave at IIT ITMO TechTalks 2013, I discussed how YSFlight Headquarters is using Amazon Web Services to give more reliable service to its users, while keeping costs down and maintenance time minimal. After discussing what I learned when using AWS for YSFHQ, I delved into how to get started with Amazon Web Services and some of the neat things you can do. -- Eric Tendian ([email protected]) Web Solutions Consultant, Tendian.io Chief Technology Officer, YSFlight Headquarters Bachelor of Info. Tech. and Mgmt., Illinois Institute of Technology (c/o 2017) Website: tendian.io
Citation preview
Flying Through the Clouds: Piloting in the World of AWS
Presented by Eric Tendian
Eric Tendian@EricTendian/in/[email protected]
ABOUT ME
Chief Technology Officer for YSFlight Headquarters
Web Solutions Consultant for Tendian.io
Hacker, tinkerer, developer
AVIATION
YSFLIGHT HEADQUARTERS
YSFLIGHT HEADQUARTERS
Online community for flight simulator pilots
Discussion forums
Based around the game YSFlight
YSFHQ.com
OUR PHILOSOPHY
Make the users work to get what they want
Bugs are fun!
Approach the cloud with caution
Windows Server FTW!
JUST KIDDING.
Our real philosophy:
Give the user the best experience, always
Pilots want fast & free
Fix, test, deploy
Use cloud whenever possible
What it really looks like
HOW DID WE ACCOMPLISH THIS?
AMAZON WEB SERVICES
http://www.youtube.com/watch?v=jOhbTAU4OPI
OUR SETUP
One EC2 micro instance
One RDS micro instance
Multiple S3 buckets
Cloudflare DNS
WHAT WE LEARNED
Start small
Make use of all AWS resources
Cloud is $$$ when managed poorly
Developers love AWS
HOW CAN I USE AWS?
Development machine(s)
Testing on the cloud
Side projects
Startups
Static website
EXCITED? LET’S GET STARTED.
Getting Started with AWSEric TendianWeb Solutions Consultant, Tendian.io
Launching aninstance
Region
Regions
Regions Region
US-WEST (N.California) EU-WEST (Ireland)
ASIA PAC(Tokyo)
ASIA PAC(Singapore)
US-WEST (Oregon)
SOUTH AMERICA (SaoPaulo)
US-EAST (Virginia)
GOV CLOUD
ASIA PAC(Sydney)
Wizard
Launch Wizard
Choose operating system
Launch Wizard
Launch!
Launch
Launch Confirmation
InstanceDNS name
Public Address
Instance DNS name
SSH
SSH
EC2
Compute
Vertical ScalingFrom $0.02/hr
Elastic Compute Cloud (EC2)Basic unit of compute capacityRange of CPU, memory & local disk options18 Instance types available, from micro to cluster compute
Feature Details
Flexible Run Windows or Linux distributions
Scalable Wide range of instance types from micro to cluster compute
Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created
Full control Full root or administrator rights
Secure Full firewall control via Security Groups
Monitoring Publishes metrics to Cloud Watch
Inexpensive On-demand, Reserved and Spot instance types
VM Import/Export Import and export VM images to transfer configurations in and out of EC2
256
128
64
32
16
8
4
2
11 2 4 8 16 32 64
EC2 Compute Units128 256
EC2 instance types
High I/O 4XL 60.5 GB35 EC2 Compute Units16 virtual cores2*1024 GB SSD-based local instancestorage
Mem
ory
(GB)
Small 1.7 GB,1 EC2 ComputeUnit1 virtual core
Micro 613 MBUp to 2 ECUs (forshort bursts)
Large 7.5 GB4 EC2 Compute Units2 virtual cores
Hi-Mem XL 17.1 GB6.5 EC2 ComputeUnits2 virtual cores
Hi-Mem 2XL 34.2 GB13 EC2 Compute Units4 virtual cores
Hi-Mem 4XL 68.4 GB26 EC2 Compute Units8 virtual cores
High-CPU Med 1.7GB5 EC2 ComputeUnits2 virtual cores
High-CPU XL 7 GB20 EC2 ComputeUnits8 virtual cores
Medium 3.7 GB,2 EC2 ComputeUnits1 virtual core
M3 XL 15 GB13 EC2 Compute Units 4 virtualcoresEBS storage only
M3 2XL 30 GB26 EC2 Compute Units 8 virtualcoresEBS storage only
Extra Large 15 GB8 EC2 ComputeUnits4 virtual cores
Cluster GPU 4XL 22 GB33.5 EC2 Compute Units,2 x NVIDIA Tesla “Fermi”M2050 GPUs
Cluster Compute 4XL 23 GB33.5 EC2 Compute Units
Cluster Compute 8XL 60.5GB88 EC2 Compute Units
High Storage 8XL 117 GB35 EC2 Compute Units,24 * 2 TB ephemeraldrives10 GB Ethernet
Hi-Mem Cluster Compute 8XL244 GB88 EC2 Compute Units16 virtual cores240 GB SSD
EC2 instance types
Light Spiky
EC2 Compute Units
Mem
ory
(GB)
SpecialStorage
Instanc e
AMI
Amazon Machine Image
Running or Stopped machine
AZ
EBS EBS EBS
Availability Zone
EBSSnapshots
S3
EBS EBS EBS
S3 Buckets
Region
EC2 terminology
More details!
Sign up:aws.amazon.com
Sign up
1 2 3 4 5
5Sign up
1 2 3 4
5Sign up
1 2 3 4
4 5Sign up
1 2 3
You will needCredit card information – you won’t pay unless you use resources
A telephone – on which to receive an automated security call
3 4 5Sign up
1 2
You will needCredit card information – you won’t pay unless you use resources
A telephone – on which to receive an automated security call
Best practiceSetup billing alerts so you can be notified when levels of spend arereached
If you have existing accounts, consider using consolidated billing to bringthem together under one payment
Sign up
1 2 3 4 5
750 hours of Amazon EC2 Linux/RedHat/Suse Micro Instance usage750 hours of Amazon EC2 Microsoft Windows Server Micro Instance usage 750 hours of an Elastic Load Balancer30 GB of Amazon Elastic Block Storage 5 GB of Amazon S3 standard storage100 MB of storage, 5 units of write capacity, and 10 units of read capacity for Amazon DynamoDB* 25 Amazon SimpleDB Machine Hours and 1 GB of Storage1,000 Amazon SWF workflow executions*1,000,000 Requests of Amazon Simple Queue Service*1,000,000 Requests, 100,000 HTTP and 1,000 email notifications for Amazon Simple Notification Service* 10 Amazon CloudWatch metrics, 10 alarms, and 1,000,000 API requests*15 GB of bandwidth out aggregated across all AWS services750 hours of Amazon RDS for SQL Server Micro DB Instance usage20 GB of RDS database storage10 million RDS I/Os20 GB of backup storage for your automated RDS database backups and any user-initiated DB Snapshots 20 minutes of SD transcoding or 10 minutes of HD transcoding in Amazon Elastic Transcoder*
Free tier http://aws.amazon.com/free/
2 3 4 5Sign up
1
Sign up
1Key pairs
2 3 4 5
Logging in to an instance
Public Key
Inserted by Amazon into each EC2 instance that
you launch
Private Key
Downloaded and stored by you
Standard SSH RSA Key pair
Public/Private Keys
Public key provided by AWS to EC2 instance for secure, personalized, initial, non-generic access
Supports NIST and other security standards for providing non-default user access
Instance key pairs
EC2Instance
Comms securedwith private key
4 5Sign up
1Key pairs
2 3
Public Key
Inserted by Amazon into each EC2 instance that
you launch
Private Key
Downloaded and stored by you
Instance key pairs
EC2Instance
Comms securedwith private key
Private keys are not stored by AWS
Standard SSH RSA Key pair
Public/Private Keys
Public key provided by AWS to EC2 instance for secure, personalized, initial, non-generic access
Supports NIST and other security standards for providing non-default user access
Sign up
1Key pairs
2 3 4 5
AWS generated keysSelect your region
Create keys
Give them a name
Private key is generated and downloaded by your browser immediately
Create 1 key pair for all resources or as many as you like (e.g 1 per server type)
Import your own keys
You supply only the public key to AWS
Sign up
1Key pairs
2 3 4 5
ssh –I [email protected]
1. Linux Launch (First Boot)
1. Instance initialization scripts insert publickey into ~/.ssh/authorized_keys
2. User connects with SSH using their PrivateKey
Sign up
1Key pairs
2 3 4 5
ssh –I [email protected]
2. User connects with SSH using their PrivateKey
You can’t log into a Linuxinstance without key
3 4 5Sign up
1Key pairs
2
1. Linux Launch (First Boot)
1. Instance initialization scripts insert publickey into ~/.ssh/authorized_keys
ssh –I [email protected]
Don’t lose it
3 4 5Sign up
1Key pairs
2
1. Linux Launch (First Boot)
1. Instance initialization scripts insert publickey into ~/.ssh/authorized_keys
2. User connects with SSH using their PrivateKey
1. Windows Launch (First Boot Sequence)
2. Instance initialization scripts:
a) Creates a random Administrator password
b) Encrypts random password with Public Key
c) Reports encrypted password to Windows System Log
3. User retrieves the encrypted password and decrypts it with their Private Key (usingAWS Console or API Call)
Sign up
1Key pairs
2 3 4 5
Choose key pair when launching instance
4 5Sign up
1Key pairs
2 3
Keepsecure
Do notshare
Rotate Need toknow
4 5Sign up
1Key pairs
2 3
Sign up
1Key pairs
2 3 4 5
Allowing accessto the instance
Sign up
1Key pairs
2Access
3 4 5
sudo yum -y install httpd
chkconfig httpd onsudo
sudo /etc/init.d/httpd start
Let’s install something
Install apache web server
Set it to run as a service
Start the web server
Sign up
1Key pairs
2Access
3 4 5
Security groups
Security Group
EC2 Classic EC2 VPC (virtual private cloud)
Inbound only Inbound and outbound
TCP, UDP, ICMP only Any protocol
Assigned at launch Assigned at launch or when running
Modify anytime Modify anytime
instance
Port 80(HTTP)
Port 22(SSH)
Name Description Protocol Port rangeIP Address, range, or another security group
Sign up
1Key pairs
2Access
3 4 5
Security groups
Added port 80to group
Open our security group
Sign up
1Key pairs
2Access
3 4 5
Test it by hitting the public DNS name of the instance
Sign up
1Key pairs
2Access
3 4 5
Sign up
1Key pairs
2Access
3 4 5
Reuse your instance!
Sign up
1Key pairs
2Access
3Image
4 5
Makes a snapshot of the instance
Creates an image that is private to you
Saves time in deployments and system setup
Sign up
1Key pairs
2Access
3Image
4 5
Create image
Sign up
1Key pairs
2Access
3Image
4 5
Name it and
create
Sign up
1Key pairs
2Access
3Image
4 5
Your AMI
Sign up
1Key pairs
2Access
3Image
4 5
…and launch a
new instance from the
AMI
Sign up
1Key pairs
2Access
3Image
4 5
Sign up
1Key pairs
2Access
3Image
4 5
Who can startan instance?
Sign up
1Key pairs
2Access
3Image
4IAM users
5
Identity and Access Management:
Securely control access to AWS services and resources for your
users
Sign up
1Key pairs
2Access
3Image
4IAM users
5
Account ownerAccess to all subscribed services Access to billing reportsAccess to console, REST and SOAP APIs
IAM users/groups
Access to specific servicesAccess to console and/or REST APIs and/or SOAP APIs
Sign up
1Key pairs
2Access
3Image
4IAM users
5
Account ownerAccess to all subscribed services Access to billing reportsAccess to console, REST and SOAP APIs
Regular usersIAM users/groups
Access to specific servicesAccess to console and/or REST APIs and/or SOAP APIs
Master user account – owns payment method
Sign up
1Key pairs
2IAM users
5Access
3Image
4
Account
Administrators Developers Applications
Bob
Kevin
Tomcat
Jim Brad
Mark
Susan
Reporting
Console
Sign up
1Key pairs
2Access
3Image
4IAM users
5
Account
Administrators Developers Applications
Bob
KevinMulti-factor authentication
Tomcat
Jim Brad
Mark
Susan
Reporting
Console
Groups
Sign up
1Key pairs
2Access
3Image
4IAM users
5
KevinAWS system entitlements
RolesAccount
Administrators Developers Applications
Bob
Tomcat
Jim Brad
Mark
Susan
Reporting
Console
Sign up
1Key pairs
2Access
3Image
4IAM users
5
{"Statement": [{
"Effect": "Allow", "Action": ["elasticbeanstalk:*", "ec2:*", "elasticloadbalancing:*", "autoscaling:*", "cloudwatch:*","s3:*","sns:*"
],"Resource": "*"
}]
}
Policy drivenDeclarative definition of rights for groups
Policies control access to AWS APIs
Sign up
1Key pairs
2IAM users
5Access
3Image
4
Sign up
1Key pairs
2Access
3Image
4IAM users
5
Next Steps
Auto ScalingAutomatic re-sizing of compute clusters based upon demand
Elastic Load BalancingCreate highly scalable applicationsDistribute load across EC2 instances in multiple availability zones
Relational Database ServiceDatabase-as-a-ServiceNo need to install or manage database instances Scalable and fault tolerant configurations
Next Steps
aws.amazon.comget started with the free tier
Thanks!Q & A?
For more info, please visit:
http://engineering.ysfhq.com
http://aws.amazon.com/