View
389
Download
0
Embed Size (px)
Citation preview
How to break apart a monolithic system safely
without destroying your teamMatthew Skelton, Skelton Thatcher Consulting
@matthewpskelton
06 June 2017, London DevOps meetup, Facebook London - #londondevops
TodayCognitive load for teams
‘Monolith’Code Forensics
Team-first boundariesMonolith-splitting recipe
For now, let’s forget:
MicroservicesCQRS / Event Sourcing
Queues(Architectural changes)
Team-first digital transformation30+ organisations
UK, US, EU, India, China
We build modern capabilities
by mentoring your teams
How to break apart a monolithic system safely without destroying your
team
Safer, more rapid changes to software systems
(Business Agility)
A ‘team-first’ approach to software subsystem
boundaries
TEAM
TEAM
capabilitiesappetite & aptitude
understanding
responsibilities
(assumption)
the team is stable, slowly changing, and long-lived
#NoProjects
Conway’s Law
“organizations which design systems ... are constrained to
produce designs which are copies of the communication structures of
these organizations”
– Mel Conway, 1968http://www.melconway.com/Home/Conways_Law.html
Front-end developers
Back-end developers
‘Reverse Conway’Tobbe Gyllebring (@drunkcod)
homomorphic force
(#Conway #Yawnoc)
HT @allankellynet
(same) (shape)
Cognitive load for teams
Cognitive load
the total amount of mental effort being used in
the working memory(see Sweller, 1988)
Cognitive load
Intrinsic Extraneous (Irrelevant )
Germane (Relevant)
‘Hacking Your Head’: Jo Pearce
See http://www.slideshare.net/JoPearce5/hacking-your-head-managing-information-overload-45-mix
@jdpearce
We have SCIENCE!
Science since 1988 (!)• Driskell et al, 1999 ‘Does Stress Lead to a Loss of Team Perspective?’ Group Dynamics:
Theory, Research, and Practice 3, no. 4 (1999): 291.
• Fan et al, 2010 ‘Learning HMM-Based Cognitive Load Models for Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119.
• Ilgen & Hollenbeck, 1993 ‘Effective Team Performance under Stress and Normal Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993.
• Johnston et al, 2002 ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002.
• Sweller, John, 1994 ‘Cognitive Load Theory, Learning Difficulty, and Instructional Design’. Learning and Instruction 4 (1994): 295–312.
• Sweller, John, 1988. ‘Cognitive Load during Problem Solving: Effects on Learning’. Cognitive Science 12, no. 2 (1988): 257–285.
“stress impacts teamperformance … by narrowing or weakening the team-level
perspective required foreffective team behavior.”
– Driskell et al, 1999Group Dynamics: Theory, Research, and Practice 1999, Vol. 3, No. 4,291-302
(not just ‘pop’ science!)
‘Monolith’
monolith
μόνο λίθος“single stone”
Reiner Flassig - CC BY-SA 2.0 de - Wikipedia
“Don’t start with a monolithwhen your goal is a
microservices architecture”
– Stefan Tilkov, innoQ
http://martinfowler.com/articles/dont-start-monolith.html
“Start monolithic and extract”
– Tammer Saleh, Pivotal
https://www.infoq.com/presentations/cloud-anti-patterns
A ‘team-first’ approach to software subsystem
boundaries
Types of software monoliths•Application monolith•Joined at the DB•Monolithic build (rebuild everything)•Monolithic releases (coupled)•Monolithic thinking (standardisation)
Application monolith
Single block of codeDeployed as a unit
Joined at the DB
Difficult to change separately (but not impossible)Risk is (probably) elevated
Chris Collyer, http://www.stone-circles.org.uk/stone/pentreifan.htm
Monolithic builds
One gigantic CI build just to get a new version of any component
Monolithic releases
Smaller components bundled together into a ‘release’
Monolithic thinking
‘One-size-fits-all’ for teamsAssumption that minimising variation is A Good Thing
Dangers of splitting a monolith•Reduced domain consistency•Data duplication (unintentional)•Additional operational complexity due to distributed system and async messaging•Degraded UX across the product
Splitting a monolith
Reiner Flassig - CC BY-SA 2.0 de - Wikipedia
Choose the right technique for splitting Understand the nature of the monolith
‘Fracture planes’ for code•Business domain bounded context•Regulatory compliance•Change cadence•Risk•Performance isolation•User personas•Team location
‘Fracture planes’ for code
Ask: Could we consume this thing as a service?
‘Fracture planes’ for code
Ask: Could we provide this thing as a service?
Code Forensics
ForensicsYour Code as a Crime SceneAdam Tornhill
Adam TornhillCode, Crime, Complexity: Analyzing software with forensic psychologyAdam TornhillTEDxTrondheim
youtube.com/watch?v=qJ_hplxTYJw
‘Code Maat’ tool
Adam Tornhill, http://www.adamtornhill.com/articles/crimescene/codeascrimescene.htm
Code City plus Code Maat forensics
Beware of badly-named subsystems
"information-poor abstract names are magnets for extra [unwanted] responsibilities"
– Adam Tornhillp.185, Your Code as a Crime Scene
Team-first boundaries
DevOpsTopologies.com
Team types
Component team
Platform / ’substrate’ team
Supporting / ‘productivity’ team
Product/Feature team
Team typesde
vops
topo
logies.com Platform / ’substrate’ team
Product/Feature team
Team typesde
vops
topo
logies.com
Component team
Platform / ’substrate’ team
Product/Feature team
Team typesde
vops
topo
logies.com
Component team
Platform / ’substrate’ team
Product/Feature team
Supporting / ‘productivity’ team
Code repositoriesAlign repositories to subsystem boundaries
Avoid monolithic-y repos like TFS** Don’t get me started on TFS, grrr…
Code repositories
Repo 1 Build Test Deploy Run
Repo 2 Build Test Deploy Run
Repo 3 Build Test Deploy Run
“You can use a monorepo only if your organisation has published a
scientific paper on Computer Science. Otherwise, use one repo
per deployable runnable thing.”
– Matthew SkeltonLondonCD meetup group, 11 Oct 2016 ☺
Find natural or available ‘fracture planes’ for splitting
a monolith
(with the team in mind)
When not to split a monolith•‘Heritage’ ERP system (‘cloudified’)•No native Unit Test framework•20-30 min startup times•VMs need 56GB RAM (yes)•CI builds take 50 mins
Monolith-splitting recipeTried and tested!
Monolith-splitting recipe
1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments
4. Split off segments one-by-one
Instrument the monolith
Instrument the monolith
Instrument the monolith
search by event
Event ID
{Delivered,InTransit,Arrived}
transaction trace
Correlation ID
612999958…
Technical
Domain
public enum EventID
{
// Badly-initialised logging data
NotSet = 0,
// An unrecognised event has occurred
UnexpectedError = 10000,
ApplicationStarted = 20000,
ApplicationShutdownNoticeReceived = 20001,
PageGenerationStarted = 30000,
PageGenerationCompleted = 30001,
MessageQueued = 40000,
MessagePeeked = 40001,
BasketItemAdded = 60001,
BasketItemRemoved = 60002,
CreditCardDetailsSubmitted = 70001,
// ...
}
BasketItemAdded = 60001
BasketItemRemoved = 60002
Instrument the monolith
Correlation ID Logs
Event ID
use logging as a channel/vector to
make distributed systems more testable
use logging as a channel/vector to
make distributed systems more testable
Grok data flows and fault responses
Grok data flows and fault responses
Correlation ID
Event ID
Unexpected collaborating subsystems
Undetected fault condition
Grok data flows and fault responses
Correlation ID
Event ID
Adjust subsystem boundaries
Fix poor fault responses
runbooktemplate.infoRun Book dialogue sheets
Align teams to available segments
Align teams to available segments
Align teams to available segmentsMap to business domain
Align teams to available segmentsIdentify likely components or ‘platform’ elements
Split off segments one-by-one
Split off segments one-by-one
Split off segments one-by-oneSeparate:- Builds- Infrastructure- Deployments- Versions- Lifecycle
Team needs / responsibilities / capabilities come first
use logging as a channel/vector to
make distributed systems more testable
Invest in Build & Release Engineering
Monolith-splitting recipe *
1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments
4. Split off segments one-by-one(5. Invest in Build & Release Engineering)
* The simplistic version
Monolith-splitting recipe *
1. Instrument 2. Grok behaviour
3. Align teams4. Split off segments
5. Invest in Build & Release* The simplistic version
Determine monolith type(Apply ‘Code Forensics’)
Find ‘fracture planes’Assess cognitive load for teams
Use the monolith-splitting recipe
How to break apart a monolithic system safely
without destroying your team
Training: 1-day workshop
Splitting monolithic software safely without destroying the team
Industry experience
Industry experienceExamples from recent work with clients
Different monolithsDifferent ‘fracture planes’
Software for R&D teamsCustomers: over 80% of top 20
global pharma companies
2014:Monolithic: builds, releasesPharma GxP: traceabilityChanges slow and tricky
Fracture plane: business domain
Fracture plane: technology
Compose with packages Logging for insights
Technology splitsImproved traceability
Major UK broadcaster (TV + online)Increasing digital delivery of content
2014:Wide mix of technologies
Need for visibility of changesNeed for rapid turnaround
Fracture planes: read-only, location & cadence
Split off Reporting (read-only)Split on change cadence
Increased change throughput
UK’s second largest credit reference agency
Customers: major banks + large orgs
2014:6-weekly monolithic releaseOutage-sensitive customers
Fracture planes: risk & business domain
Split releases based on riskAlign teams to value streamsMore rapid release cadence
Software for Internal CommsCustomers: FTSE 100 & Fortune 1000
2015:Successful system (monolithic)
ISO 27001 complianceAim to expand the org (x N)
Fracture plane: business domain
Optimise split for team engagement & personas
Minimise ISO 27001 footprintAided organisation expansion
Determine monolith type(Apply ‘Code Forensics’)
Find ‘fracture planes’Assess cognitive load for teams
Use the monolith-splitting recipe
How to break apart a monolithic system safely
without destroying your team
Training: 1-day workshop
Splitting monolithic software safely without destroying the team
Further material
Books & articles• Working Effectively with Legacy Code, by Michael Feathers• Building Microservices by Sam Newman (O’Reilly, 2015)• Managing Cognitive Load for Team Learning by Jo Pearce
http://12devsofxmas.co.uk/2015/12/day-3-managing-cognitive-load-for-team-learning/
• Continuous Delivery with Windows and .NET by Matthew Skelton and Chris O’Dell (O’Reilly, 2016) http://cdwithwindows.net/
• Team Guide to Software Operability by Matthew Skelton and Rob Thatcher (Skelton Thatcher Publications, 2016) http://operabilitybook.com/
• Ye Olde Monolith by Pter Pilgrim https://dzone.com/articles/ye-olde-monolith
Training• From Monolith to Microservices (online training) – Sam Newman,
author of Building Microserviceshttp://www.oreilly.com/live-training/from-monolith-to-microservices.html
• Run Book template & Run Book dialogue sheets http://runbooktemplate.info/
Talks & slides• Hacking Your Head – Managing Information Overload, by Jo
Pearce - http://www.slideshare.net/JoPearce5/hacking-your-head-managing-information-overload-45-mix /http://conferences.oreilly.com/oscon/open-source-eu/public/schedule/detail/53013
• Principles of Microservices by Sam Newman @ Devoxx 2015 https://www.youtube.com/watch?v=PFQnNFe27kU
Research papers• Driskell, James E., Eduardo Salas, and Joan Johnston. ‘Does Stress Lead to a Loss of Team
Perspective?’ Group Dynamics: Theory, Research, and Practice 3, no. 4 (1999): 291.• Fan, Xiaocong, Po-Chun Chen, and John Yen. ‘Learning HMM-Based Cognitive Load Models for
Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119. • Ilgen, Daniel R., and John R. Hollenbeck. ‘Effective Team Performance under Stress and Normal
Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA284683.
• Johnston, Joan H., Stephen M. Fiore, Carol Paris, and C. A. Smith. ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA525820.
• Sweller, John. ‘Cognitive Load Theory, Learning Difficulty, and Instructional Design’. Learning and Instruction 4 (1994): 295–312.
• Sweller, John. ‘Cognitive Load during Problem Solving: Effects on Learning’. Cognitive Science 12, no. 2 (1988): 257–285.
Determine monolith type(Apply ‘Code Forensics’)
Find ‘fracture planes’Assess cognitive load for teams
Use the monolith-splitting recipe
How to break apart a monolithic system safely
without destroying your team