Microservices at Amazon Chris Munns Business Development Manager - DevOps

About me: Chris Munns - munns@amazon.com, @chrismunns

– Business Development Manager – DevOps – New Yorker – Previously:

•  AWS Solutions Architect 2011-2014 •  Lead of Infrastructure/DevOps @hingeapp •  Formerly on operations teams @Etsy and @Meetup •  Little time at a hedge fund, Xerox and others

– Rochester Institute of Technology: Applied Networking and Systems Administration ’05

– Internet infrastructure geek

The Micro-elephant in the room

https://www.flickr.com/photos/davidrosenphotography/14711027415/

Microservices “is a software architecture style in which complex applications are composed of small, independent processes communicating with each other using

language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to

system-building.” - Wikipedia

https://en.wikipedia.org/wiki/Microservices

Microservices Smart people are talking about it: • Sam Newman - book “Building

Microservices” O’Reilly Publishing • Adrian Cockcroft – numerous talks,

presentations, blog posts “Loosely coupled service oriented

architecture with bounded contexts” • Martin Fowler – numerous blog posts •  2 sessions at AWS Re:Invent 2014 •  ~8 sessions at AWS Re:Invent 2015

But is it just a fad? Phil Calcado(ex-SoundCloud) in his Sept 8th post entitled “How we ended up with microservices.” highlighted the recent growth of the term:

Data from Google Trends http://philcalcado.com/2015/09/08/how_we_ended_up_with_microservices.html

But is it just a fad? As they say in dramatic police dramas….Let’s ENHANCE!:

Data from Google Trends

Lots of folks are doing it • Gilt: “From Monolith Ruby App to Distributed Scala Micro-

Services” NYC Tech Talks • Nike: “Nike’s Journey to Microservices” – AWS Re:Invent

2014 • SoundCloud: ”Building Products at SoundCloud—Part III:

Microservices in Scala and Finagle” • Capital One: “Lack Of Legacy Lets Capital One Build

Nimble Infrastructure” – ThePlatform.net • Hailo: “A Journey into Microservices” – sudo.hailoapp.com

Brilliant folks are sharing tools for it • Netflix: Dozens of tools - https://netflix.github.io/ • Airbnb: SmartStack - http://nerds.airbnb.com/smartstack-service-discovery-cloud/

• Twitter: Mesos - https://mesos.apache.org/ • LinkedIn: Rest.li - https://github.com/linkedin/rest.li

Monolith development lifecycle

developers

release test build

delivery pipeline app

•  Single-purpose

•  Connect only through APIs

•  Connect over HTTPS •  Largely “black

boxes” to each other

•  “Microservices”

Microservices vs. SOA Wikipedia: “distinct from a Service-oriented architecture (SOA) in that the latter aims at integrating various (business) applications whereas several microservices belong to one application only” Martin Fowler: “This common manifestation of SOA has led some microservice advocates to reject the SOA label entirely, although others consider microservices to be one form of SOA” Oracle: “The picture that emerges is not of microservices as an alternative to SOA, but rather as a way to restore flexibility that may have been lost in SOAs that became too rigid and monolithic.”

https://en.wikipedia.org/wiki/Microservices http://martinfowler.com/articles/microservices.html#MicroservicesAndSoa

http://www.oracle.com/technetwork/issue-archive/2015/15-mar/o25architect-2458702.html

Microservices: •  Many very small components •  Business logic lives inside of

single service domain •  Simple wire protocols(HTTP

with XML/JSON) •  API driven with SDKs/Clients

SOA: •  Fewer more sophisticated

components •  Business logic can live across

domains •  Enterprise Service Bus like

layers between services •  Middleware

Microservices vs. SOA

14

•  Two-pizza teams

•  Full ownership

•  Full accountability

•  Aligned incentives

•  “DevOps”

Wait, now we’re talking about

Pizza? 16 ©2015 Apigee. All Rights Reserved. Image By: Chris Munns – munns@amazon.com

How do Two Pizza Teams work?

We call them “Service teams” • Own the “primitives” they build:

– Product planning (roadmap) – Development work – Operational/Client support work

• “You build it, you run it” • Part of a larger concentrated org (Amazon.com, AWS, Prime, etc)

17

Who Does QA?

18

The Two Pizza Team

Who Does On Call?

19 Image By: Chris Munns – munns@amazon.com

The Two Pizza Team

What does Ops Do?

20

Not Exist

What about Ops/QA/Etc?

Everyone exists on a “service team” focused on their primitive(s): • SDE’s focused on developing • PM’s focused on product direction • TPM’s help drive development • SE’s focused on infra/tooling • SDET’s focused on test excellence throughout the organization

Some folks are shared across the org, some on individual teams

21

Boy, that sounds like a lot of freedom?

It is! Teams are empowered and also held to high standards: • Thorough onboarding/training • Patterns/practices defined at scale and with 20+ years of organizational knowledge

• Regular technical and business metric reviews • Regular sharing of new tools, services, technologies, etc, by internal subject matter experts

22

Missing tools

developers delivery pipeline services

???

•  Self-service

•  Technology-agnostic

•  Encourage best practices

•  Single-purpose services

•  Deployment service

•  No downtime deployments

•  Health checking

•  Versioned artifacts and rollbacks

•  Continuous delivery

•  From check-in to production

•  Automated actions and transitions

•  >90% of teams

Pipelines

Microservice development lifecycle

developers delivery pipelines services

release test build

release test build

release test build

release test build

release test build

release test build

= 50 million deployments a year

Thousands of teams × Microservice architecture

× Continuous delivery × Multiple environments

I’m sold, how do I make my own

pizza? 29 ©2015 Apigee. All Rights Reserved. Image By: Chris Munns – munns@amazon.com

How to make pizza?

• Yeast • Water • Sugar • Olive oil • Tomato sauce • Cheese • <insert favorite toppings here>

30

No Chris, I mean in the organizational/

technological sense 31 ©2015 Apigee. All Rights Reserved. Image By: Chris Munns – munns@amazon.com

The keys to your own Two Pizza teams & Microservices

Culture Practices Tools

32

The keys to your own Two Pizza teams & Microservices Culture

– Ownership & Accountability go hand in hand – Larger teams typically move slower than smaller teams – Insist on standards of excellence, but be flexible on the how

Practices – Continuous Integration & Delivery – Simplify operational tasks

Tools – CI & CD tools – Infrastructure management – Metrics/Monitoring/Logging – Communication and Collaboration

33

The keys to your own Two Pizza teams & Microservices Culture

– Ownership & Accountability go hand in hand – Larger teams typically move slower than smaller teams – Insist on standards of excellence, but be flexible on the how

Practices – Continuous Integration & Delivery – Simplify operational tasks

Tools – CI & CD tools – Infrastructure management – Metrics/Monitoring/Logging – Communication and Collaboration

34

The keys to your own Two Pizza teams & Microservices Culture

– Ownership & Accountability go hand in hand – Larger teams typically move slower than smaller teams – Insist on standards of excellence, but be flexible on the how

Practices – Continuous Integration & Delivery – Simplify operational tasks

Tools – CI & CD tools – Infrastructure management – Metrics/Monitoring/Logging – Communication and Collaboration

35

The keys to your own Two Pizza teams & Microservices Culture

– Ownership & Accountability go hand in hand – Larger teams typically move slower than smaller teams – Insist on standards of excellence, but be flexible on the how

Practices – Continuous Integration & Delivery – Simplify operational tasks

Tools – CI & CD tools – Infrastructure management – Metrics/Monitoring/Logging – Communication and Collaboration

36

Internal “Service teams” should own, support, train, drive excellence

of these tools

Apollo and Pipelines as AWS Services:

CodePipeline launched July 2015

CodeDeploy launched Nov 2014

Establishing a pattern for services and clients

It’s important that the organization isn’t reinventing the wheel on every new service: •  How are clients going to communicate? •  What cross service authorization requirements are there? •  How do services prevent abuse? •  How do you quickly build clients against a service? •  How do services handle discovery of others services and resources?

Use an API Gateway!

Internet

Mobile Apps

Websites

Services

API Gateway

AWS Lambda functions

API Gateway Cache

Endpoints on Amazon EC2

Any other publicly accessible endpoint

Amazon CloudWatch Monitoring

Amazon CloudFront

HOST METRICS

SERVICE METRICS

LOG ANALYSIS

BUILD METRICS

FIN, ACK This is a big topic that is hard to cover in a single session, day, week! Key pointers: • Microservices patterns are changing how we are building APIs

and our applications in general • Team structure plays an incredible role in building, running,

scaling Microservices • Focusing on flexible tooling that provides standards can help an

organization focus on what matters • Enable the smart people you hire to do great things by giving

them both ownership and accountability on those things success

41

Thank you

42