AWS re:Invent 2016: Getting Started with Serverless Architectures (CMP211)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Jeremy Edberg, Founder and CEO, MinOps

December 2016

CMP211

Getting Started

with Serverless Architectures

Test and prod are different

Prod is in need of constant

updates

Slow iteration and deployment

Polyglot unfriendly

Deploy in weeks, live for years

Physical Servers

Prod is immutable

Rapid iteration and deployment

Multi-tenancy

Polyglot friendly

Deploy in minutes, live for weeks

Virtual Machines

Test and prod are the same

Prod is immutable

Rapid(er) iteration and deployment

High multi-tenancy

Polyglot friendly

Deploy in seconds, live for hours

Containers

Smallest unit of compute

Super scalable

Rapid iteration

Extreme multi-tenancy

Very polyglot friendly

Easier to collaborate

Deploy independently, live for

seconds

Serverless

A whole lot of choices

Amazon ecosystemHodgepodge of services

A whole lot of choices

AWS Serverless

Ecosystem

Lambda

SNS

DynamoDB

SQS

S3

Amazon Kinesis

Continuous

scaling

No servers to

manage

Never pay for

idle – No cold

servers (only

happy

accountants)

Benefits of AWS Lambda

Lambda is a core

component of modern

software

What is serverless

anyway?

• There are still servers, you just

don’t manage them anymore

• It also means you don’t access

them anymore

• So you don’t need to (or get to)

optimize them.

What is Serverless

Computing?

• VMs

• Machine as the unit of scale

• Abstracts the hardware

• Containers

• Application as the unit of scale

• Abstracts the OS

• Serverless

• Functions as the unit of scale

• Abstracts the language runtime

EC2

ECS

Lambda

How do I choose?

• VMs

• “I want to configure machines,

storage, networking, and my OS”

• Containers

• “I want to run servers, configure

applications, and control scaling”

• Serverless

• “Run my code when it’s needed”

EC2

ECS

Lambda

What if I buy my own

software?

Lambda is so flexible you can add it in or around existing software

Serverless computing is

all about speeding up

development by allowing

rapid iteration and

removing management

overhead

A brief journey through history

80’s Mainframe Cycle

• Cost $1MM to $100MM

• 1 to 5 years to execute

• Usually had to bet the whole

company

• Cost of failure -- Bankruptcy

90‘s Client/Server Cycle

• Cost $100K to $10MM

• 3 to 6 months to execute

• Usually had to bet a product or

division

• Cost of failure -- Revenue hit, CXX

job

00‘s Agile Cycle

• Cost $10K to $1MM

• 3 to 6 weeks to execute

• Bet a product feature

• Cost of failure -- product manager

reputation

Product Manager

Developer

QA

Ops

BI / Analytics

Cost and size

of risk

decreased

Rate of

change

increased

ReleaseTestBuild

Developer Deployment Pain: High

ReleaseTestBuild

ReleaseTestBuild

ReleaseTestBuild

ReleaseTestBuild

ReleaseTestBuild

Developer Deployment Pain: Medium

ReleaseTestBuild

ReleaseTestBuild

ReleaseTestBuild

ReleaseTestBuild

ReleaseTestBuild

Developer Deployment Pain: Low

Multiple services

Internal Microservices Platform

Monolithic

Success follows a standard

pattern

Why Microservices?

Distributed Computing and a

Distributed Workforce

• The two go hand in hand

when you have a good

distributed systems

culture

• Microservices and micro

teams

Cloud Native

10s of thousands of instances,

thousands created and removed

daily

Thousands of storage nodes,

petabytes of data, nodes can

be removed without harm

(Some folks call this microservices)

Movie Ratings

Personalization Engine User Info

Movie Metadata

Similar Movies

ReviewsA/B Test Engine

Discovery

API

Streaming

API

Movie Ratings

Personalization Engine User Info

Movie Metadata

Similar Movies

Reviews

A/B Test Engine

Discovery

API

Streaming

API

Content

EncodingCDN

Management

QOS

LoggingDRM

OpenConnect

Edge

Locations

Browse

Play

Watch

• Services are built by different

teams who work together to

figure out what each service

will provide.

• The service owner publishes

an API that anyone can use

and returns proper response

codes

Highly aligned,

loosely coupled

Developers own their product

from beginning to end

If the customer isn’t happy,

the developer shouldn’t be happy

Freedom and

Responsibility

Proper

Microservices

Architecture

Service

and

Resource

Discovery

Network

and

Traffic

Config

Automated

Testing

Continuous

DeploymentSecurity

Monitoring

and

Alerting

Mature

companies

spend 25% of

their engineering

resources on

their internal

platform

And they’re all building the

same thing…

Wasting time and effort they could

be spending on product innovation

Proper

Microservices

Architecture

Service

and

Resource

Discovery

Network

and

Traffic

Config

Automated

Testing

Continuous

DeploymentSecurity

Monitoring

and

Alerting

And when you’re done, it is

only “good enough”

Building an internal

microservices platform is hard

So why go Serverless?

Proper

Microservices

Architecture

Service

and

Resource

Discovery

Network

and

Traffic

Config

Automated

Testing

Continuous

DeploymentSecurity

Monitoring

and

Alerting

Proper

Microservices

Architecture

Automated

Testing

Continuous

DeploymentSecurity

Security

• Shorter TTL ==

less chance for

an attack to take

hold

The Microservices Architecture

Microservices Management

Tools

Web servers

Code Libraries

Frameworks

Config Management

API Management

Deployment, CI/CDContainers

etc.

etc.

etc.

What do all the parts of

microservices have in common?

Servers

Capacity planning

Right-sizing

Automatic scaling

Load and performance

Patches

Tuning

Configuration

Utilization

Access control

Packages and AMIs

Serverless

Right-sizing

Automatic scaling

Load and performance

Patches

Tuning

Configuration

Utilization

Access control

Packages and AMIs

Fully managed

Continuous Scaling

Function is the deployment unit

Capacity planning

Serverless ==

Microservices -

ManagementProvisioning Scaling

Microservice problems we still have

Queue length

Persistent storage

Fallbacks and graceful

degradation

Monitoring and alerting

Integration and deployment

Testing

What does Lambda do for you?

• Scales server capacity

automatically

• API to trigger execution

• Ensures function is executed in

parallel and at scale

• Logging, monitoring, etc.

• Easy pricing

Monitoring• Everything is in CloudWatch or CloudWatch logs

Pricing

• Choose your RAM

from 128 MB to

1500 MB

• CPU and network

scaled based on

RAM

Cost Comparison

Cost Comparison

There’s about 2.5M seconds in a month, so 3M requests is about 1.2 per second

The T2.Small is $18.98 a month, more than Lambda already

Pay per Request

• Buy compute in 100ms

increments

• Low request charge

• No minimums

• No per-device fees

Never pay for idle time!

Free Tier

1,000,000 requests and 400,000

GB every month, every

customer

How does one use ?

Author your code

• AWS SDK built-in

• Access to /tmp,

processes,

threads, etc.

• Node.js, Python

• Java and any JVM

language

• Use any library you

want

Choose an event source

• S3

• Amazon

Kinesis

• SNS

• DynamoDB

• CloudWatch

• Config Rules

• Amazon Echo

• API Gateway

• Lambda

functions

• SES

• Cognito

• CloudFormation

• CodeCommit

• Echo

• Scheduled

events

Choose your resources

• Select from 128 MB to 1.5 GB

• CPU and network are chosen

automatically

• Pay only for what you actually

use

Choose your

authorization model

• Securely grant access to

resources and VPC

• Fine-grained control over

who calls your functions

with IAM

Choose a Network

• Default

• All functions have

internet access

• No access to your

other VPCs

• Still in a VPC

• Customer VPC

• Private communication

within your VPC

• Same configuration as

your other VPCs (ENI,

route table, security

groups, NAT)

Deploy your code

• Use the GUI or

• Upload to S3

or Lambda

directly

• Use a third

party tool

Write the function

Create the IAM role

Add extra permissions for a push model

Zip the function and dependencies

Test the function with mock data

Add an event source to the function

View the output of the live function

Write the function

Create the IAM role

Add extra permissions for a push model

Zip the function and dependencies

Test the function with mock data

Add an event source to the function

View the output of the live function minops.com/oss

Use a tool

PolishDeep

Integration

Language

Support

API Gateway

SupportNotes

Kappa Meh Best Meh No

Best for

Python, same

author as boto

Apex Great Ok Best NoTies into

Terraform

Serverless

(Jaws)Great Good Good Yes

Uses Cloudformation

Chalice Great Ok Only Python YesGreat for

REST APIs

Serverless Ex

pressGreat

Only w/ API

GatewayOnly NodeJS Yes Built by AWS

Create a unified API

frontend for multiple

microservices

DDoS protection and

throttling for backend

systems

Authenticate and

authorize requests

Benefits of Amazon API

Gateway

Lambda Use Cases

• Application Backend

• Data Processing

• Command and Control

• Any functional or event

based system

Data Processing

Upload

to S3

S3 Bucket Lambda

S3 Notification

Compressed

Image

Compress Image

Upload back to S3

Serverless Website

Static

Content

Dynamic

Content

API Gateway Lambda DynamoDB

Internet

Mobile Apps

Websites

Services

AWS Lambda

functions

API Gateway

Cache

Endpoints on

Amazon EC2

Any other publicly

accessible endpointAmazon

CloudWatch

Amazon

CloudFront

Amazon

API Gateway

Serverless API w/ API

Gateway

API Gateway Lambda SNS

Mobile Backend

Kinesis Lambda DynamoDB

Real-Time Analytics Engine

Live video stream processing

CloudFront S3

Ingest

Lambda functions

HQ Copy

480p

Transcode

360p

Transcode

Audio Only

Transcode

S3

Playback

CloudFront

streaming

Thumbnails

Word Generator

• Generate some number of English

looking words using an ngram

database

• Allow a prefix to be specified

https://github.com/jedberg/wordgen

Alexa and Lambda Demo

From conception to prod

in 5 minutes

Codebase

Structure

Lambda lets you manage

your code and infrastructure

in the same place

Tips and Tricks

Immutable Data

• If you can, write your

software such that

everything in the cache is

immutable.

Moving data is

the single biggest cost

your distributed system

will incur

But you need

to move data for

reliability,

so it’s a tradeoff

Use queues

as often as possible

0

10

20

30

40

50

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Cumulative Flow Diagram

Items

Seconds

Arrivals

Departures

Avoiding Infinite Loops

• With a distributed

team, this is an easy

mistake to make

• To avoid it, pass a call

stack and check for

self in the stack

Store your data properly

• No local instance

access

• Store everything in

S3, DynamoDB, or

ElastiCache

DynamoDB S3

Function Schedules• Schedule a function to keep it warm

if traffic is low

• Schedule a function to poll your

SQS queues or use a DynamoDB

table as a queue and trigger on put

• Schedule a function with a more

granular timer if one minute isn’t

granular enough

• Have one function call another to

get more timers

Function Versions

• Use the environment variable

service for mutable

configuration

• To rollback, switch to an old

alias or an old API Gateway

collection

• Use a traffic shaping function

for Blue / Green deployments

• Point directly at an ARN to

lock to a particular version

Lambda VPC Basics• VPC is always on, but with

sane defaults

• Lambda functions can

access your private

resources if added to your

VPC

• But they will lose internet

access unless you have a

managed NAT, regardless

of your internet gateway or

permissive security group

rules

Lambda VPC Basics

• Make sure you have

enough ENIs for max

concurrency

• And enough IPs in your

pool

• And enough per AZ

Amazon API Gateway

Best Practices• Use request/response

mapping templates

everywhere within reason,

not passthrough.

• Take ownership of HTTP

response codes

• Use Swagger

import/export for cross-

account sharing

• Use Mock integrations

• Combine with Cognito for

managed end user-based

access control.

• Use stage variables (inject API

config values into Lambda

functions for logging, behavior)

Additional Best

Practices• Use strategic, consumable

naming conventions (Lambda

function names, IAM roles, API

names, API stage names, etc.)

• Use naming conventions and

versioning to create

automation.

• Externalize authorization to

IAM roles whenever possible

• Least privilege and separate IAM roles

• Externalize configuration –DynamoDB is great for this.

• Contact AWS Support before known large scaling events

• Be aware of service throttling, engage AWS Support if so.

Thank you!

Remember to complete

your evaluations!

Questions?

Email: anything@jedberg.net

Twitter: @jedberg

Web: www.jedberg.net

Facebook: facebook.com/jedberg

Linkedin:

www.linkedin.com/in/jedberg

Company: minops.com