Microservices lessons from trenches

Microservices: lessons from the trenches

Team Lead at ThoughtWorksMehdi Khalili

twitter: @MehdiKhalili

http://www.mehdi-khalili.com/presentations/microservices-lessons-dddmel/

http://www.mehdi-khalili.com/microservices-lessons/

Microservices isAWESOME!!

Benefits of microservices

• Scalability• Loose Coupling• Fault Tolerance• Scalable Development• Technical Diversification• Continuous Delivery

http://giphy.com/gifs/parkour-tJLFLygAbaxYk

http://giphy.com/gifs/fail-parkour-dZeSENj3pXT6o

Challenges

• Fault Tolerance• Loose Coupling• Greenfield• Culture

Fault Tolerance

http://giphy.com/gifs/fail-parkour-movement-7SqLFyxYHal6o

30 services with 99.9% up-time 43m downtime per service a month1296m downtime a monthone day of downtime a month

The Akamai story!

contain the failure

Netflix: assumed broken components

Reading

• caching• data duplication!• graceful degradation

Actions and events

• guaranteed delivery• eventual consistency

be proactive about failure

The story of recycling app pool!

• monitoring endpoints• synthetic monitoring

be proactive about failure

troubleshooting

The story of missing param!

• extensive logging• correlation id• log aggregation

troubleshooting

Correlation Id Service From Timestamp Message

996a3975-4996

Registration::Create

nil 02 Aug 2014 19:55:15 AEST +10:00

bbbe64ab-c84c Cart::Get nil 02 Aug 2014 19:55:15 AEST +10:00

996a3975-4996

Accounting::Create


02 Aug 2014 19:55:16 AEST +10:00

996a3975-4996

Avatar::Create Registration::Create

02 Aug 2014 19:55:16 AEST +10:00

NullReference


troubleshooting


996a3975-4996


nil 02 Aug 2014 19:55:15 AEST +10:00


996a3975-4996

Accounting::Create


02 Aug 2014 19:55:16 AEST +10:00

996a3975-4996


02 Aug 2014 19:55:16 AEST +10:00

NullReference


troubleshooting


996a3975-4996


nil 02 Aug 2014 19:55:15 AEST +10:00


996a3975-4996

Accounting::Create


02 Aug 2014 19:55:16 AEST +10:00

996a3975-4996


02 Aug 2014 19:55:16 AEST +10:00

NullReference

Loose Coupling

http://giphy.com/gifs/fail-parkour-2BuleA8z0C4uY

you need the big picture

The soft-delete story!


beautiful diagrams!


Correlation Id

Service From Timestamp Message

0a669422-5e44


nil 02 Aug 2014 19:55:15 AEST +10:00

bbbe64ab-c84c

Accounting::Create

User::AddAccount 01 Aug 2014 05:10:01 AEST +10:00

996a3975-4996

Accounting::Create


02 Aug 2014 19:55:16 AEST +10:00

e30e4e37-b1e1


02 Aug 2014 19:55:16 AEST +10:00

NullReference

catch contract violation at build time not run-time

consumer

provider

consumer

consumer driven contracts

catch contract violation at build time


consumer

provider

consumer

service libraries

service lib

service lib


Greenfield Microservices

http://giphy.com/gifs/zSmGb2eYIAx3y

you need well defined service boundaries

greenfield leads tounknown boundaries

The nano-services story!

nanoservice

kill

me

now

monolithSize

pain

always start with monolith

extract the bits that would benefit from

microservices

every greenfield project is a

startup“a human institution designed

to create a new product or service under conditions of

extreme uncertainty”

Minimum Viable Product

“that version of a new product which allows a team to collect

the maximum amount of validated learning about

customers with the least effort”

microservices is hard toimplementand deployand monitor

Microservices is NOT MVP

never start withmicroservices

The Culture

http://giphy.com/gifs/fail-parkour-jCi66sK0E770s

nothing can save you from cultural failure

“our release process has 43 manual and approval steps”

“fixing the bug took me 5 minutes,releasing it to prod took 3 weeks!”

heavy processes

“test automation might work for others but our system is different.”

“we have 30 developers and 85 testers!”

no automation

“we need to stop obsessing about quality and focus on getting this

out soon.”

no respect for quality

“we need a DevOps team!”

“shortcut” to happiness

test automationdeployment automation

infrastructure automationDevOps culture

The Culture

Netflix:” you code it, you'll deploy

it

and you'll be on pager-duty.”

How do you sell microservices?

DO NOT

takeaways

Fault tolerance

• Contain the failure• Caching, data duplication and graceful

degradation• Guaranteed delivery and eventual consistency

• Be proactive about failure• Monitoring

• Use extensive logging

Loose coupling

• You need the big picture• Self documenting application

• Build-time contract verification• Consumer Driven Contracts• Service Libraries

Avoid greenfield microservices

• Start with monolith• Learn your domain model• Microservices is NOT MVP

takeaway

• Culture, culture, culture!• Test automation• Deployment automation• Infrastructure automation• DevOps culture

microservices isawesome

benefits vs overheadsScalability Operations &

monitoringLoose coupling Contract violation

Fault tolerance Fault isolationScalable

developmentCommunication

overheadLower cognitive load Losing the big picture

Tech diversification Too many things to learn

Continuous delivery DevOps culture

do it to the extent its benefits outweigh

overheads and challenges

Next time you want to do microservices

• Do you actually need it?• Do you need it now?• Are the challenges worth the

benefits?• Does it fit your CULTURE?

Resources

Thank you

twitter: @MehdiKhalili

http://www.mehdi-khalili.com/presentations/microservices-lessons-dddmel/

http://www.mehdi-khalili.com/microservices-lessons/

Technology

Microservices lessons from trenches