Microservices the Good Bad and the Ugly

Microservices The Good, Bad and Ugly

Adrian Cockcroft @adrianco Technology Fellow - Battery Ventures

June 2015

When microservices are good and when you don’t need them What new problems they bring

What does @adrianco do?

@adrianco

Technology Due Diligence on Deals

Presentations at Conferences

Presentations at Companies

Technical Advice for Portfolio Companies

Program Committee for Conferences

Networking with Interesting PeopleTinkering with

Technologies

Maintain Relationship with Cloud Vendors

Key Goal of the CIO?

Key Goal of the CIO?Align IT with the business


Note: Netflix doesn’t have a CIO…


Note: Netflix doesn’t have a CIO…Product related IT is embedded in the business - totally aligned

Value Chain Mapping

Simon Wardley http://blog.gardeviance.org/2014/11/how-to-get-to-strategy-in-ten-steps.html Related tools and training http://www.wardleymaps.com/

http://www.wardleymaps.com/

Value Chain Mapping


Your unique product - Agile


Value Chain Mapping


Your unique product - Agile Best of breed as a Service - Lean


Value Chain Mapping


Your unique product - Agile

Undifferentiated utility suppliers - 6sigma

Best of breed as a Service - Lean


Product Development

Processes

Assumption: Process prevents

problems

Organizations build up slow complex “Scar

tissue” processes

Observe

Orient

Decide

Act Continuous Delivery

Observe

Orient

Decide

Act

Land grab opportunity Competitive

Move

Customer Pain Point

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

INNOVATION

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

Model Hypotheses

INNOVATION

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

Model Hypotheses

BIG DATA

INNOVATION

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

JFDI

Plan Response

Share Plans

Model Hypotheses

BIG DATA

INNOVATION

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

JFDI

Plan Response

Share Plans

Model Hypotheses

BIG DATA

INNOVATION

CULTURE

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

JFDI

Plan Response

Share Plans

Incremental Features

Automatic Deploy

Launch AB Test

Model Hypotheses

BIG DATA

INNOVATION

CULTURE

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

JFDI

Plan Response

Share Plans


Automatic Deploy

Launch AB Test

Model Hypotheses

BIG DATA

INNOVATION

CULTURE

CLOUD

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

JFDI

Plan Response

Share Plans


Automatic Deploy

Launch AB Test

Model Hypotheses

BIG DATA

INNOVATION

CULTURE

CLOUD

Measure Customers

Continuous Delivery

Observe

Orient

Decide

Act


Move

Customer Pain Point

Analysis

JFDI

Plan Response

Share Plans


Automatic Deploy

Launch AB Test

Model Hypotheses

BIG DATA

INNOVATION

CULTURE

CLOUD

Measure Customers

Continuous Delivery

Breaking Down the SILOs


QA DBA Sys Adm

Net Adm

SAN AdmDevUXProd

Mgr


QA DBA Sys Adm

Net Adm

SAN AdmDevUXProd

Mgr

Product Team Using Monolithic DeliveryProduct Team Using Monolithic Delivery


QA DBA Sys Adm

Net Adm

SAN AdmDevUXProd

MgrProduct Team Using Microservices

Product Team Using Monolithic Delivery

Product Team Using Microservices




QA DBA Sys Adm

Net Adm

SAN AdmDevUXProd



Platform TeamProduct Team Using Microservices




QA DBA Sys Adm

Net Adm

SAN AdmDevUXProd



Platform TeamA P IProduct Team Using Microservices




QA DBA Sys Adm

Net Adm

SAN AdmDevUXProd



Platform Team

DevOps is a Re-Org!

A P IProduct Team Using Microservices



Release Plan

Developer

Developer

Developer

Developer

Developer

QA Release Integration

Ops Replace Old With New

Release

Monolithic service updates

Works well with a small number of developers and a single language like php, java or ruby

Release Plan

Developer

Developer

Developer

Developer

Developer



Release

Bugs



Release Plan

Developer

Developer

Developer

Developer

Developer



Release

Bugs

Bugs



Use monolithic apps for small teams, simple systems and when you must, to optimize for efficiency and latency

Developer

Developer

Developer

Developer

Developer

Old Release Still Running

Release Plan

Release Plan

Release Plan

Release Plan

Immutable microservice deployment scales, is faster with large teams and diverse platform components

Developer

Developer

Developer

Developer

Developer


Release Plan

Release Plan

Release Plan

Release Plan

Deploy Feature to Production





Developer

Developer

Developer

Developer

Developer


Release Plan

Release Plan

Release Plan

Release Plan





Bugs


Developer

Developer

Developer

Developer

Developer


Release Plan

Release Plan

Release Plan

Release Plan





Bugs



Configure

Configure

Developer

Developer

Developer

Release Plan

Release Plan

Release Plan

Deploy Standardized

Services

Standardized portable container deployment saves time and effort

https://hub.docker.com


Configure

Configure

Developer

Developer

Developer

Release Plan

Release Plan

Release Plan

Deploy Standardized

Services




Bugs


Standardized portable container deployment saves time and effort



Developer Developer

Run What You Wrote

Developer Developer

Developer Developer

Run What You Wrote

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Developer Developer

Developer Developer

Run What You Wrote

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Developer Developer

Monitoring Tools

DeveloperDeveloper Developer

Run What You Wrote

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Developer Developer

Monitoring Tools


Run What You Wrote

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Developer Developer

Site Reliability

Monitoring Tools

Availability Metrics

99.95% customer success rate


Run What You Wrote

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Developer Developer

Manager Manager

Site Reliability

Monitoring Tools




Run What You Wrote

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Micro service

Developer Developer

Manager Manager

VP Engineering

Site Reliability

Monitoring Tools



Change One Thing at a Time!

What Happened?Rate of change

increased

Cost and size and risk of change

reduced

Low Cost of Change Using Docker

Developers • Compile/Build • Seconds

Extend container • Package dependencies • Seconds

PaaS deploy Container • Docker startup • Seconds

Low Cost of Change Using Docker

Fast tooling supports continuous delivery of many tiny changes

Developers • Compile/Build • Seconds

Extend container • Package dependencies • Seconds

PaaS deploy Container • Docker startup • Seconds

Disruptor: Continuous Delivery with

Containerized Microservices

Microservices

A Microservice Definition

Loosely coupled service oriented architecture with bounded contexts



If every service has to be updated at the same time it’s not loosely coupled



If every service has to be updated at the same time it’s not loosely coupled

If you have to know too much about surrounding services you don’t have a bounded context. See the Domain Driven Design book by Eric Evans.

Coupling Concerns

http://en.wikipedia.org/wiki/Conway's_law

●Conway’s Law - organizational coupling

●Centralized Database Schemas

●Enterprise Service Bus - centralized message queues

●Inflexible Protocol Versioning

http://en.wikipedia.org/wiki/Conway's_law

Non-Destructive Production Updates

● “Immutable Code” Service Pattern

● Existing services are unchanged, old code remains in service

● New code deploys as a new service group

● No impact to production until traffic routing changes

● A|B Tests, Feature Flags and Version Routing control traffic

● First users in the test cell are the developer and test engineers

● A cohort of users is added looking for measurable improvement

● Finally make default for everyone, keeping old code for a while

Speeding Up Deployments

Datacenter Snowflakes • Deploy in months • Live for years



Virtualized and Cloud • Deploy in minutes • Live for weeks




Container Deployments • Deploy in seconds • Live for minutes/hours





AWS Lambda Events • Respond in milliseconds • Live for seconds


Measuring CPU usage once a minute makes no sense for containers… Coping with rate of change is the first challenge for monitoring tools.




AWS Lambda Events • Respond in milliseconds • Live for seconds

Inspiration

http://www.infoq.com/presentations/scale-gilt

http://www.slideshare.net/mcculloughsean/itier-breaking-up-the-monolith-philly-ete

http://www.infoq.com/presentations/Twitter-Timeline-Scalability http://www.infoq.com/presentations/twitter-soa

http://www.infoq.com/presentations/Zipkin

https://speakerdeck.com/mattheath/scaling-micro-services-in-go-highload-plus-plus-2014

State of the Art in Web Scale Microservice Architectures

AWS Re:Invent : Asgard to Zuul https://www.youtube.com/watch?v=p7ysHhs5hl0 Resiliency at Massive Scale https://www.youtube.com/watch?v=ZfYJHtVL1_w

Microservice Architecture https://www.youtube.com/watch?v=CriDUYtfrjs

http://www.infoq.com/presentations/scale-gilt

http://www.slideshare.net/mcculloughsean/itier-breaking-up-the-monolith-philly-ete

http://www.infoq.com/presentations/Twitter-Timeline-Scalability

http://www.infoq.com/presentations/twitter-soa

http://www.infoq.com/presentations/Zipkin

https://speakerdeck.com/mattheath/scaling-micro-services-in-go-highload-plus-plus-2014

https://www.youtube.com/watch?v=p7ysHhs5hl0

https://www.youtube.com/watch?v=ZfYJHtVL1_w

https://www.youtube.com/watch?v=CriDUYtfrjs

Microservice Concerns

ConfigurationTooling Discovery Routing Observability

Development: Languages and Container

Operational: Orchestration and Deployment Infrastructure

Datastores

Microservices

Edda Archaius

Configuration

Asgard Aminator

Tooling

Eureka Prana

Discovery

Denominator Zuul, Netty Ribbon 2.0

Routing

Hystrix Pytheus SALP

Observability

Java, Groovy, Scala, Clojure, Python, Node.js with AMI and Docker Containers

Manual Orchestration with Asgard and deployment on AWS or Eucalyptus

Ephemeral datastores using Dynomite, Memcached, Astyanax, Staash, Priam, Cassandra

Microservices

Edda Archaius

Configuration

Asgard Aminator

Tooling

Eureka Prana

Discovery

Denominator Zuul, Netty Ribbon 2.0

Routing

Hystrix Pytheus SALP

Observability

Java, Groovy, Scala, Clojure, Python, Node.js with AMI and Docker Containers

Manual Orchestration with Asgard and deployment on AWS or Eucalyptus

Ephemeral datastores using Dynomite, Memcached, Astyanax, Staash, Priam, Cassandra

Focus on global distribution, high scale and availability

NetflixOSS Components• See “Zero to Docker” to try out Ne1lixOSS in seconds… • Security and access management setup -‐ Security Monkey • Account Management: Asgard to deploy & Ice for cost monitoring • Build Tools: Aminator to automate baking AMIs • Service Registry and Searchable Account History: Eureka & Edda • ConfiguraOon Management: Archaius dynamic property system • Data storage: Cassandra, Astyanax, Priam, EVCache, SURO logging to KaUa/ElasOc • Dynamic traffic rouOng: Denominator, Zuul, Ribbon, Karyon • Availability: Simian Army (Chaos Monkey), Hystrix, Turbine, Atlas monitoring • Developer producOvity: Blitz4J, GCViz, Pytheas dashboards, RxJava • Big Data: Genie for Hadoop PaaS, LipsOck visualizer for Pig, Pigpen for Clojure • Apps to get started: SpringCloud, RSS Reader, ACME Air, FluxCapacitor

Twitter Microservices

Decider

ConfigurationTooling

Finagle Zookeeper

Discovery

Finagle Netty

Routing

Zipkin

Observability

Scala with JVM Container

Orchestration using Aurora deployment in datacenters using Mesos

Custom Cassandra-like datastore: Manhattan

Twitter Microservices

Decider

ConfigurationTooling

Finagle Zookeeper

Discovery

Finagle Netty

Routing

Zipkin

Observability

Scala with JVM Container

Orchestration using Aurora deployment in datacenters using Mesos

Custom Cassandra-like datastore: Manhattan

Focus on efficient datacenter deployment at scale

Challenges for Microservices

Managing Scale

A Possible Hierarchy Continents

Regions Zones

Services Versions

Containers Instances

How Many? 3 to 5

2-4 per Continent 1-5 per Region 100’s per Zone

Many per Service 1000’s per Version

10,000’s

It’s much more challenging than just a large number of

machines

Flow

Some tools can show the request flow

across a few services

But interesting architectures have a lot of microservices! Flow visualization is

a big challenge.

See http://www.slideshare.net/LappleApple/gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture

Failures

ELB Load Balancer

Zuul API Proxy

Karyon Business Logic

Staash Data Access Layer

Priam Cassandra Datastore

Simple NetflixOSS style microservices architecture on three AWS Availability Zones

ELB Load Balancer

Zuul API Proxy





ELB Load Balancer

Zuul API Proxy





Zone partition/failure What should you do? What should monitors show?

ELB Load Balancer

Zuul API Proxy






By design, everything works with 2 of 3 zones running. This is not an outage, inform but don’t touch anything! Halt deployments perhaps?

ELB Load Balancer

Zuul API Proxy






By design, everything works with 2 of 3 zones running. This is not an outage, inform but don’t touch anything! Halt deployments perhaps?

Challenge: understand and communicate common microservice failure patterns.

Testing

Testing monitoring tools at scale gets expensive quickly…

Simulation

Simulated Microservices

Model and visualize microservices Simulate interesting architectures Generate large scale configurations Eventually stress test real tools

See github.com/adrianco/spigo Simulate Protocol Interactions in Go Visualize with D3

ELB Load Balancer

Zuul API Proxy




Three Availability Zones

http://github.com/adrianco/spigo

Definition of an architecture

{ "arch": "netflixoss", "description":"A very simple Netflix service. See http://netflix.github.io/ to decode the package names", "version": "arch-0.0", "victim": "homepage", "services": [ { "name": "cassSubscriber", "package": "priamCassandra", "count": 6, "regions": 1, "dependencies": ["cassSubscriber", "eureka"]}, { "name": "evcacheSubscriber","package": "store", "count": 3, "regions": 1, "dependencies": []}, { "name": "subscriber", "package": "staash", "count": 6, "regions": 1, "dependencies": ["cassSubscriber", "evcacheSubscriber"]}, { "name": "login", "package": "karyon", "count": 18, "regions": 1, "dependencies": ["subscriber"]}, { "name": "homepage", "package": "karyon", "count": 24, "regions": 1, "dependencies": ["subscriber"]}, { "name": "wwwproxy", "package": "zuul", "count": 6, "regions": 1, "dependencies": ["login", "homepage"]}, { "name": "www-elb", "package": "elb", "count": 0, "regions": 1, "dependencies": ["wwwproxy"]}, { "name": "www", "package": "denominator", "count": 0, "regions": 0, "dependencies": ["www-elb"]} ] }

Header includes chaos monkey victim

New tier name

Tier package

Region count: 1

Node count

List of tier dependencies

How does Spigo work?

Creates and animates microservices Single Go program on this laptop Simulates 100,000+ instances Dynamic Go channels replace tcp/ip “gotocol” messages simulate http etc. About 250,000 messages/sec Supports social network architecture LAMP or NetflixOSS architectures

Spigo Nanoservice Structurefunc Start(listener chan gotocol.Message) { ... for { select { case msg := <-listener: switch msg.Imposition { case gotocol.Hello: // get named by parent ... case gotocol.NameDrop: // someone new to talk to ... case gotocol.GetRequest: // upstream request handler ... case gotocol.GetResponse:// downstream response handler ... case gotocol.Goodbye: // tell parent I’m going away now gotocol.Message{gotocol.Goodbye, nil, time.Now(), name}.GoSend(parent) return } case <-eurekaTicker.C: // poll the service registry ... } } }

Why Build Spigo?

Generate test microservice configurations at scale Stress monitoring tools display capabilities

Eventually (i.e. not implemented yet) Dynamically vary configuration: autoscale, code push Chaos monkey for microservice, zone, region failures D3 websocket dynamic browser interface

Conference Driven Development…

OSCON Microservices Workshop

What’s Next?

Developer Concerns Agile, Lean & Rugged

(Faster, Cheaper and Safer) See GOTO London Sept 2015

Forward Thinking

Forward Thinking

Any Questions?

Disclosure: some of the companies mentioned may be Battery Ventures Portfolio Companies See www.battery.com for a list of portfolio investments

● Battery Ventures http://www.battery.com ● Adrian’s Tweets @adrianco and Blog http://perfcap.blogspot.com ● Slideshare http://slideshare.com/adriancockcroft

● Monitorama Opening Keynote Portland OR - May 7th, 2014 ● GOTO Chicago Opening Keynote May 20th, 2014 ● Qcon New York – Speed and Scale - June 11th, 2014 ● Structure - Cloud Trends - San Francisco - June 19th, 2014 ● GOTO Copenhagen/Aarhus – Fast Delivery - Denmark – Sept 25th, 2014 ● DevOps Enterprise Summit - San Francisco - Oct 21-23rd, 2014 #DOES14 ● GOTO Berlin - Migrating to Microservices - Germany - Nov 6th, 2014 ● AWS Re:Invent - Cloud Native Cost Optimization - Las Vegas - November 14th, 2014 ● O’Reilly Software Architecture Conference - Fast Delivery, Monitoring Challenge - Boston March 16th 2015

http://www.battery.com

http://perfcap.blogspot.com

http://slideshare.com/adriancockcroft



Security

Visit http://www.battery.com/our-companies/ for a full list of all portfolio companies in which all Battery Funds have invested.

Palo Alto Networks

Enterprise IT

Operations & Management

Big DataCompute

Networking

Storage

http://www.battery.com/our-companies/