Download pdf - Berlin - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/AWS Summit Berlin 2015... · • Appropriate VF driver ... Intel® 82599 Virtual Function driver . Enhanced Networking

Berlin

Amazon EC2 Deep Dive Michael Hanisch

@hanimic #awssummit

Let’s get started…

Amazon EC2 instances

AMIs & Virtualization Types

EBS-backed AMIs

Physical host server

instance

AMI

snapshot New root

volume

S3-backed AMIs


instance

AMI

disk

Copy image data to instance store

Mount volume

EBS-backed vs. S3-backed AMIs

•  EBS faster to boot (esp. with EBS GP2) •  Need at least one root volume per instance •  S3-backed AMIs require instance storage

Only M3 / I2 / D2 / G2 instance types + previous generation

Virtualization Primer: x86 CPU

•  Prior to Intel VT-x –  Binary translation –  Para-virtualization (PV)

•  After Intel VT-x –  Hardware assisted virtualization (HVM)

•  PV vs. HVM Amazon Machine Images •  vCPU scheduling choices and variability

Virtualization Primer: I/O and Devices

•  Scheduling I/O requests between virtual devices and shared physical hardware

•  Split driver model for shared devices; requires host resources

•  Intel VT-d –  Direct pass through and IOMMU for dedicated devices

Virtualization types: PV vs. HVM

Application

Bare Metal

Kernel

Application

Kernel

PV

VMM

•  PV-HVM allows Application to call directly into the Kernel •  PV requires going through the VMM – overhead! •  Applications that are system call bound are most affected

Kernel Application

VMM

PV-HVM


•  PV / “paravirtual” AMIs –  Need customized OS kernel that supports virtualization –  Can use separate AKI (Amazon Kernel Image) and ARI

(Amazon Ramdisk Image), ie. Drop-in replacements for Linux kernel + drivers without changing the AMI

–  Special PV-GRUB bootloader

Support for PV AMIs

Previous Generation Instance Types T1 M1 M2 C1 CC2 CR1 HI1 HS1 CG1

Support for PV AMIs

Current Generation Instance Types T2 M3 M4 C3 C4 R3 I2 D2 G2


•  HVM AMIs –  Guest OS can run unmodified –  No custom Kernels etc. needed –  Bootloader starts off of MBR of root device –  Use PV drivers for fast access to underlying hardware without

translation (“PV-HVM”) ! Faster network & storage IO ! Direct access to network interfaces, GPUs etc,

–  Allows for access to CPU hardware features

Support for HVM AMIs

Previous Generation Instance Types T1 M1 M2 C1 CC2 CR1 HI1 HS1 CG1

Support for HVM AMIs


The Bottom Line

Use 64 bit HVM EBS-backed AMIs (…unless you really want to use previous generation instance types)

Instance Types

c3.large Instance family

Instance generation

Instance size

CPU Storage Memory

Network

3 C

4 C

CPU

C4 Instance Type

•  Compute-Optimized Instance Family –  Highest amount of compute performance per $ –  Smaller amount of RAM per vCore

•  Custom Intel CPU (Intel Xeon E5-2666v3) •  Base speed 2.9Ghz – Intel ® Turbo Boost ≤ 3.5Ghz •  AVX2 supports 256-bit integer vectors and can process 32

single precision or 16 double precision floating point operations per cycle

•  Available in 5 different sizes –  4 to 36 vCores –  3.75 GiB to 60 GiB RAM

C4 Instance Type

•  Only 2 vCores can run at highest Turbo Boost speed at the same time, others run at base freq

•  Alternatively, all vCores can run at 3.2GHz •  Influence speed using Processor State Control •  Access to CPU features: AVX 2.0, AES-NI

C4 Instance Type

•  C4.8xlarge allows to control the CPU’s C-state and P-state –  Control which idle state vCPUs can reach (C0=running, C6=off) –  Control the available performance state for vCPUs

•  Decrease latency of thread switching •  Max. frequency for two threads vs. higher base

performance •  Turn off turbo mode for more consistent performance

3 M

4 M

M4 Instance Type

•  General Purpose Instance Family –  Less CPU per RAM than C4 –  More RAM per vCore than C4 (and less than R3)

•  Custom Intel CPU (Xeon E5-2676 v3) •  Base speed 2.4Ghz – turbo up to 3.0Ghz

(2.70 GHz when all cores are running) •  Available in 5 different sizes

–  2 to 40 vCores –  8 GiB to 160 GiB RAM

2 T

T2 Instance Type

•  General Purpose Instance Family •  Burstable CPU •  Intel Xeon CPU •  Base 2.5Ghz, Intel ® Turbo Boost up to 3.3Ghz •  Available in 4 different sizes

–  1 to 2 vCores –  1 to 8 GiB RAM

T2 Instance Type

•  General Purpose Instance Family •  Burstable CPU •  Intel Xeon CPU •  Base 2.5Ghz, Intel ® Turbo Boost up to 3.3Ghz •  Available in 4 different sizes

–  1 to 2 vCores –  1 to 8 GiB RAM

Max. credit depends on instance size Initial credit after boot!

An instance earns CPU credits at a steady rate

Understanding T2 CPU Credits

37

1 CPU Credit provides the performance of a full CPU core for 1 minute

An instance consumes credits when active

Model vCPU CPU credits/h Memory (all) Storage t2.micro 1 6 1 EBS Only T2.small 1 12 2 EBS Only T2.medium 2 24 4 EBS Only T2.large 2 48 8 EBS Only

CPU Storage Memory

Network


Amazon EBS only

Amazon EBS only

Amazon EBS only

Storage Options


Amazon EBS only

Amazon EBS only

Amazon EBS only

Amazon EBS

Amazon EBS

Amazon EBS

Amazon EBS

Amazon EBS

Amazon EBS

Storage Options

VPC-only Instance Types


•  Enhanced Networking / SR-IOV •  Multiple Elastic Network Interfaces •  Multiple IP addresses •  •  More powerful Security Groups (ingress/egress) •  VPC flow-logs …

VPC-only Features

Enhanced Networking / SR-IOV

•  Single Root I/O Virtualization •  Supports significantly higher # of packets / sec •  Reduced latency, less jitter


•  C4, M4, C3, R3, D2, I2 instance families •  HVM virtualization type •  VPC-only •  Required kernel version

–  Linux: 2.6.32+ –  Windows: Server 2008 R2+

•  Appropriate VF driver –  Linux: ixgbevf 2.14.2+ module –  Windows: Intel® 82599 Virtual Function driver


•  Verify Intel ixgbevf driver is installed and enabled $"ethtool"(i"eth0"driver:"ixgbevf(version:"2.14.2+amzn"…"


•  Verify Intel ixgbevf driver is installed and enabled •  Enable sriov instance attribute $"aws"ec2"modify(instance(attribute"\""""((instance(id=i(XXXXXXX"\"((sriov(net(support=simple;""(Or launch from an AMI that has this flag set!)

instance

instance

instance

instance

instance

instance

instance

instance

Placement Groups •  10Gb/s network •  Fully bisected •  Low latency

Placement Groups

•  Use 10Gbps instance size for max throughput •  Enable Enhanced Networking to reduce latency

and increase packets-per-second •  Use Jumbo Frames (9001 MTU) to achieve

maximum throughput

Support for Placement Groups

Previous Generation Instance Types T1 M1 M2 C1 CC1 CC2 HI1 HS1 CG1

Support for Placement Groups


Dedicated Instances

•  Not a performance optimization! •  Only instances of same AWS AccountID are

placed on same physical server

•  Use this for compliance or security reasons not to get extra bandwidth / CPU / …

Amazon Elastic Block Store

instance volume

volume

Which volume type? How many volumes? RAID, JBOD, LVM?

Amazon Elastic Block Store


Hypervisor

Virt. block device

Amazon EBS

EBS Server


Hypervisor

Virt. block device

Amazon EBS

EBS Server

EBS traffic only! Other traffic

Secondary EBS Server

Amazon EBS Physical host server

Hypervisor

Virt. block device Amazon EBS

Primary

Amazon EBS Performance

Depends on a number of factors: •  EBS Volume type

–  IO capacity (IOPS) –  Volume bandwidth

•  Instance (network) bandwidth –  Instance type & size –  EBS-optimized option enabled or not

Amazon EBS Volume Types & Limits

General Purpose (SSD) •  1 GiB ≤ size ≤ 16 TiB •  10,000 IOPS •  Max. 160 MiB/s •  Up to 256 KB / IOP

Provisioned IOPS •  4 GiB ≤ size ≤ 16 TiB •  20,000 IOPS •  Max. 320 MiB/s •  Up to 256 KB / IOP

Amazon EC2 instance types & limits

•  Instance bandwidth –  Varies depending on instance size –  Depends on utilization of network link (internet traffic, …)

•  EBS optimized –  Supported on most instance families –  Additional reserved bandwidth of up to 4 GiB/s for

communication with Amazon EBS volumes –  Up to 32,000 IOPS @ 16kB, ie. 256 MB/s throughput

Amazon EC2 instance types & limits

•  Instance throughput –  Up to 48,000 IOPS (16 kB) –  Up to 800 MB/s

The End

What is still missing?

•  VPC features & usage scenarios •  Monitoring & alarming •  Resilience & Recovery •  Instance store performance •  Purchase models: Spot / On Demand / RI …