94
Strategic management of Technical Debt Philippe Kruchten April 4 th , 2017 ICSA 2017, Göteborg

Strategic management of Technical Debt - Philippe · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Embed Size (px)

Citation preview

Page 1: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

StrategicmanagementofTechnicalDebt

PhilippeKruchten

April4th,2017ICSA2017,Göteborg

Page 2: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PhilippeKruchten,Ph.D.,P.Eng.,CSDP

ProfessorofSo)wareEngineeringNSERCChairinDesignEngineeringDepartmentofElectricalandComputerEngineering

UniversityofBriJshColumbiaVancouver,[email protected]

FounderandpresidentKruchtenEngineeringServicesLtdVancouver,BCCanada [email protected]@pbpk

Page 3: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Outline

•  Whatistechnicaldebt?•  Thetechnicaldebtlandscape•  Limitsofthemetaphor•  Managingtechnicaldebt•  Toolsandtechniques•  FricJoninsoUwaredevelopment•  PracJcalsteps

Page 4: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt

•  ConceptintroducedbyWardCunningham•  OUenmenJoned,rarelystudied•  AllexperiencedsoUwaredevelopers“feel”it.•  Dragslong-livedprojectsandproductsdown

Page 5: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Originofthemetaphor•  WardCunningham,atOOPSLA1992

“ShippingfirstJmecodeislikegoingintodebt.Ali^ledebtspeedsdevelopmentsolongasitispaidbackpromptlywitharewrite…Thedangeroccurswhenthedebtisnotrepaid.Everyminutespentonnot-quite-rightcodecountsasinterestonthatdebt.EnJreengineeringorganizaJonscanbebroughttoastand-sJllunderthedebtloadofanunconsolidatedimplementaJon,object-orientedorotherwise.”

Cunningham,OOPSLA1992

Page 6: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt(S.McConnell)•  Implementedfeatures(visibleandinvisible)=assets=non-debt

•  Type1:unintenJonal,non-strategic;poordesigndecisions,poorcoding

•  Type2:intenJonalandstrategic:opJmizeforthepresent,notforthefuture.–  2.Ashort-term:paidoffquickly(refactorings,etc.)

•  Largechunks:easytotrack•  Manysmallbits:cannottrack

–  2.Blong-termMcConnell2007

Page 7: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt(M.Fowler)

Fowler2009,2010

Page 8: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TimeisMoney(I.Gat)

•  Convertthisinmonetaryterms:“ThinkoftheamountofmoneytheborrowedJmerepresents–thegrandtotalrequiredtoeliminateallissuesfoundinthecode”

Gat2010

Page 9: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechDebt(JimHighsmith)

Source:Highsmith,2009

Page 10: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt(S.McConnell)

•  TD:AdesignorconstrucJonapproachthatisexpedientintheshorttermbutthatcreatesatechnicalcontextinwhichthesameworkwillcostmoretodolaterthanitwouldcosttodonow

McConnell2011

Page 11: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebtDefiniJonInsoUware-intensivesystems,technicaldebtisthecollecJonofdesignorimplementaJonconstructsthatareexpedientintheshortterm,butsetupatechnicalcontextthatcanmakefuturechangesmorecostlyorimpossible.TechnicaldebtpresentsanactualorconJngentliabilitythatimpactsinternalsystemqualiJes,primarilymaintainabilityandevolvability.

“ManagingTechnicalDebtinSoUwareEngineering,”DagstuhlReports,Vol.6,Issue4.h^p://www.dagstuhl.de/16162.

Page 12: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

FirstmorecapabiliJes

Firstmoreinfrastructure

Then,moreinfrastructure

Then,morecapabiliJes

underesJmatedre-architecJngcosts

neglectedcostofdelaytomarket

needtomonitortechnicaldebttogaininsightintolife-cycleefficiency

Example

Ozkaya,SEI,2010

Page 13: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

MakingHardChoicesaboutTechnicalDebt

Inthequesttobecomemarketleader,playersracetoreleaseaqualityproducttothemarketplace.

TheHardChoicesgameisasimulaJonofthesoUwaredevelopmentcyclemeanttocommunicatetheconceptsofuncertainty,risk,opJons,andtechnicaldebt.

HardChoicesStrategyGametoCommunicateValueofArchitectureThinkinggamedownloadablefromh^p://www.sei.cmu.edu/architecture/tools/hardchoices/.

Page 14: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg
Page 15: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PlayingtheHardChoicesBoardGame

• Thegoalofthegameistoaccumulatethemostpoints.• PlayersaccumulatepointsbycrossingENDaheadoftheircompeJtorsandcollecJngToolCards.• Thepersonwiththemostpointswins.

• MarketLeaderPoints.ThefirstplayertocrossENDgets7points,secondgets3points,andthirdgets1point.Thelastplayerremainingontheboardgetsnopoints.

•  ToolPoints.Youget1pointforeachToolCard.

Page 16: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

RulesofPlay

• StartofPlay–  Rollthedietodeterminewhogoesfirst.

Playproceedsclockwise.•  PlayerMovement

–  Duringaturn,rollthedie:MoveyourpiecethenumberofspacesindicatedonthedieminusthenumberofpenalJesincurred,determinedbythenumberofyourBridgeCards.

–  YoumaymoveineitherdirecJon,orinbothdirecJons,withinaturn.ThisincreasesyouropportunitytolandonaToolSquare.

–  Onceaplayerhasreachedtheend,noonecanmovebackwards.•  EndofPlay

–  ToentertheENDcellyoumayrollanythingequalorgreaterthanthenumberofremainingsquares.

–  Thegameendswhenoneplayerremainsontheboard.

Page 17: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SpecialSquares

• HardChoicesSquares–  WhencrossingaHardChoicesSquare,you

mustdecidewhethertogoovertheShortcutBridgeortogothelongwayandtrytocollectmoreToolCards.

• BridgesandBridgeCards–  Bridgescountasonemovement.

–  WhencrossingaShortcutBridge,youmustcollectaBridgeCard.EachBridgeCardsubtractsonefromsubsequentrollsofthedie.

–  YoumaygetridofaBridgeCardbyskippingaturnanyJmeduringthegame.

•  ToolSquaresandToolCards–  WhenlandingonaToolSquare,youmayelecttotakeaToolCard.

–  YoumayonlycollectoneToolCardforagivensquare.

–  IfyouhaveaToolCard,youmayelectnottotakenottotakeanother.Insteadyoumayplaythecard(returningittothedeck)andgetafreeturn.

Page 18: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

DebriefAUertheGame• Whatjusthappened?

–  Howdidyouexperiencethegame?–  Whatstrategiesdidyouemploy?

• Sowhat?–  Howdoyourexperiencesinthegamerelatetothestrategiesyou

employduringsoUwaredevelopmentinthefaceofuncertainty?–  Howdoesthisrelatetothechoicesyoumake—ininvesJngeffortto

gainanadvantageorpayingapricetotakeshortcuts?–  WhataretheirimplicaJons?

• Nowwhat?–  Whatwillyoutakeawayfromtheexperience?–  Whatmightyoudodifferentlyasaresult?

Page 19: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Outline

•  Whatistechnicaldebt?•  Thetechnicaldebtlandscape•  Limitsofthemetaphor•  Managingtechnicaldebt•  Toolsandtechniques•  FricJoninsoUwaredevelopment•  Furtherresearchontechnicaldebt

Page 20: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Visible

Newfeatures

Techno

logicalgap

ArchitecturaldebtStructuraldebt Codesmells

DefectsLowinternalquality

AddiJonalfuncJonality Lowexternalquality

Mostlyinvisible

TestdebtDocumentaJondebt

EvoluJonissues:evolvability Qualityissues:maintainability

Visible

architecture code

Codecomplexity

CodingstyleviolaJons

Zürich,May2012

Page 21: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg
Page 22: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg
Page 23: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TD:negaJvevalue,invisible

NewfeaturesAddedfuncHonality

Architectural,Structuralfeatures

Defects TechnicalDebt

Visible Invisible

PosiJveValue

NegaJveValue

Page 24: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Interests•  Inpresenceoftechnicaldebt,costofaddingnewfeaturesishigher;velocityislower.

•  Whenrepaying(fixing),addiJonalcostforretrofiungalreadyimplementedfeatures

•  Technicaldebtnotrepaid=>leadtoincreasedcost,forever

•  Costoffixing(repaying)increasesoverJmeM.Fowler,2009

Page 25: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Wherethemetaphorbreaks

•  IniJalinvestmentatT0inanenvironmentE0.NowinT2,EhaschangedtoE2,amismatch,hasoccurred,whichcreatesadebt.– Thedebtiscreatedbythechangeofenvironment.TherightdecisionintherightenvironmentatsomeJmemayleadtotechnicaldebt.

•  Prudent,inadvertent

Page 26: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Wherethemetaphorbreaks…

•  Technicaldebtdependsonthefuture•  Technicaldebtcannotbemeasured•  Youcanwalkawayfromtechnicaldebt•  Technicaldebtshouldnotbecompletelyeliminated

•  TechnicaldebtcannotbehandledinisolaJon•  Technicaldebtcanbeawiseinvestment

Page 27: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt(1)

1212

a

$15

$5

12

b

$16

$3

12 $18

$20 $19 $18

Page 28: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt(2)

1212

a

$15

$5

12

b

$16

$3

12 $18

8 8 $5 8 $8 8 $10

$25 $27 $28

Page 29: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TechnicalDebt(3)

1212

a

+$2

$5

12 $18

8 8 $5

$30

Page 30: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PotenJalvs.actualdebt

•  PotenJaldebt– Type1:OKtodowithtools(seeGat&co.approach)

– Type2:structural,architectural,ortechnologicalgap:Muchharder

•  Actualdebt– Whenyouknowthewayforward

K.Schmid2013

Page 31: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TDlitmustest

•  Ifyouarenotincurringanyinterest,thenitprobablyisnotadebt

McConnell2013

Page 32: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Technicaldebtasaninvestment?

Page 33: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TDandRealOpJons

P1: S0

Marketlovesit+$4M

Markethatesit+$1M

S1

NPV(P1)=-2M+0.5x4M+0.5x1M=0.5M

-2M

Source:K.Sullivan,2010atTDWorkshopSEI6/2-3

Page 34: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TDandRealOpJons(2)

P2: S0

Marketlovesit

Markethatesit+$1M

Sd

NPV(P2)=-1M+0.5x3M+0.5x1M=1M

-1M

Source:K.Sullivan,2010

-1MS1 +4M

TakingTechnicalDebthasincreasedsystemvalue.

Page 35: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TDandRealOpJons(3)

P2: S0

Marketlovesit

Markethatesit+$1M

Sd

NPV(P3)=-1M+0.67x2.5M+0.33x1M=1M

-1M

-1.5MS1 +4M

MorerealisJcally:Debt+interestHighchancesofsuccess

TakeDebt

Repaydebt

Page 36: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TDandRealOpJons(3)

P2: S0

Marketlovesit

Markethatesit+$1M

Sd

NPV(P3)=-1M+0.67x2.5M+0.33x1M=1M

-1M

-1.5MS1 +4M

MorerealisJcally:Debt+interestHighchancesofsuccess

Higherchanceofsuccess

Repaydebt+50%interest

Page 37: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TDandRealOpJons(4)

S0

Favourable

Unfavourable

Sd

S1 S2

S2d

…..

…..

Notdebtreally,butopHonswithdifferentvalues…Dowewanttoinvestinarchitecture,intest,etc…

Addfeature

?

Source:K.Sullivan,2010

Page 38: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Outline

•  Whatistechnicaldebt?•  Thetechnicaldebtlandscape•  Limitsofthemetaphor•  Managingtechnicaldebt•  Toolsandtechniques•  FricJoninsoUwaredevelopment•  Furtherresearchontechnicaldebt

Page 39: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Howdopeople“tackle”technicaldebt

Page 40: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

TacklingTechnicalDebt

Autude,approachesfound:1.  Ignoranceisbliss2.  Theelephantintheroom3.  Bigscary$$$$numbers4.  Fivestarranking5.  ConstantreducJon6.  We’reagile,soweareimmune!

Page 41: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Ignoranceisbliss

You’rejustslower,andslower,butyoudonotknowit,ordonotknowwhy

0

2

4

6

8

10

12

1 2 3 4 5 6 7

FuncHo

nalreq

uiremen

tdelivered

IteraHons

Velocity accumulatedtechnicaldebtimpactsabilitytodeliver

Page 42: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Theelephantintheroom

•  Manyintheorg.knowabouttechnicaltech.

•  Indifference:it’ssomeoneelse’sproblem

•  OrganizaJonbrokendowninsmallsilos

•  Norealwholeproductmentality

•  Short-termfocus

Page 43: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Bigscary$$$$numbers

•  Codesmells 167persondays•  Missingtest 298persondays•  Design 670persondays•  DocumentaJon 67persondaysTotalsWork 1,202personxdaysCost $577,000

Page 44: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

StaJcanalysis+ConsulJng

•  Cu^erConsorJum:Gat,etal.– UseofSonar,etc.– Focusedoncodeanalysis– TD=totalvalueoffixingthecodebase

•  CASTsoUware•  ThoughtWorks

DebtanalysisengagementsDebtreducJonengagements

Page 45: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Issues•  Fitsthemetaphor,indeed.•  LooksveryobjecJve…but…•  SubjecJvein:

– Whatiscounted– Whattooltouse–  Costtofix

NotallfixeshavethesameresulJngvalue.Sunkcostareirrelevant,lookintothefutureonly.Whatdoesitmeantobe“Debtfree”??

Page 46: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Fivestarranking

•  Definesomemaintainabilityindex•  BenchmarkrelaJvetoothersoUwareinthesamecategory

•  Re-assessregularly(e.g.,weekly)•  Lookattrends,correlatechangeswithrecentchangesincodebase

•  SIG(SoUwareImprovementGroup),Amsterdam•  Powerfultoolbehind

Page 47: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

ConstantdebtreducJon

•  Maketechnicaldebtavisibleitemonthebacklog

•  MakeitvisibleoutsideofthesoUwaredev.organizaJon

•  IncorporatedebtreducJonasaregularacJvity

•  UsebufferinlongertermplanningforyetunidenJfiedtechnicaldebt

•  Lie(?)

Page 48: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Bufferfordebtrepayment

SimpleworkEsJmateuncertainJes

DefectcorrecJon

DebtRepayment

Page 49: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Alaterrelease

Page 50: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Weareagile,sowe’reimmune!

Insomecasesweareagileandthereforewerunfasterintotechnicaldebt

Page 51: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Agilemo^os

•  “Deferdecisiontothelastresponsiblemoment”•  “YAGNI”=YouAin’tGonnaNeedIt

–  Butwhenyoudo,itistechnicaldebt–  TechnicaldebtoUenistheaccumulaJonoftoomanyYAGNIdecisions

•  “We’llrefactorthislater”•  “Delivervalue,early”•  Againthetensionbetweentheyellowstuffandthegreenstuff

•  You’resDllagilebecauseyouaren’tsloweddownbyTDyet.

Page 52: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Storyofafailure•  Largere-engineeringofacomplexdistributedworld-widesystem;2millionsLOCinC,C++,CobolandVB

•  MulJplesites,dozensofdatarepositories,hundredsofusers,24hoursoperaJon,mission-criJcal($billions)

•  xP+Scrum,1-weekiteraJons,30thenupto50developers

•  Rapidprogress,earlysuccess,featuresaredemo-able•  Directaccessto“customer”,etc.•  Aposterprojectforscalableagiledevelopment

Page 53: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Hiungthewall•  AUer4½months,difficulJes

tokeepwiththe1-weekiteraJons

•  RefactoringtakeslongerthanoneiteraJon

•  ScrapandreworkraJoincreasesdramaJcally

•  Noexternallyvisibleprogressanymore•  IteraJonsstretchedto3weeks•  Staffturn-overincreases•  Projectcomestoahalt•  Lotsofcode,nocleararchitecture,noobviouswayforward

Page 54: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

IsraelGat,2010h^p://theagileexecuJve.com/2010/09/20/how-to-break-the-vicious-cycle-of-technical-debt/

(more)RelentlessPressure

TakeTechnicalDebt

FailtoPayback

Technicaldebt

TechnicalDebtAccrues

ReducedDevelopment

TeamVelocity

Gat’sTechDebtviciouscycle

Page 55: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Value,Quality,Constraints

•  Value=extrinsicquality– Metric:Netpresentvalue

•  Quality=intrinsicquality– Metric:Technicaldebt

•  Constraints=cost,schedule,scope– Metric:Cost

Value

Quality

Cost

Highsmith2010

Page 56: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

EvoluJonoverJme

Gat&Heintz,Cu[er,2010

Page 57: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

CogniJvebiasesandTD

•  EscalaJonofcommitment– Aka,toomuchinvestedtoquit

•  Sunkcostfallacy– aka.throwinggoodmoneyaUerbad

•  Anchoring

•  Confirmatorybias

Page 58: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

32

Timeline

1

ti tj

Incurred Symptom

IntenJonalandstrategic

PayoffRework

4

5

What is the debt? Technical debt item description Risk analysis, Development state analysis

How does debt accumulate? Static and architecture analysis

When to pay back debt? Architecture-focused release planning

Page 59: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Outline

•  Whatistechnicaldebt?•  Thetechnicaldebtlandscape•  Limitsofthemetaphor•  Managingtechnicaldebt•  Toolsandtechniques•  FricJoninsoUwaredevelopment•  Furtherresearchontechnicaldebt

Page 60: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

ToolsandTechniques

Someexamples

Page 61: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

ToolsforTechnicalDebtAnalysis•  Vendorsinclude

•  CAST•  Inspearit•  SonarSource(Sonarqube)•  Thoughtworks•  SoUwareImprovementGroup(SIG)•  Laux•  Hello2morrow•  Tocéa(ScerJfy)•  Xdepend•  Klocwork•  JetBrains

Page 62: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

•  RealOpJontheory•  DependencyStructureMatrix

– PropagaJoncost•  Sonarqube•  SQALE•  ScerJfy

Page 63: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SQALE

•  SQALE=SoUwareQualityAssessmentbasedonLifecycleExpectaJon

•  Jean-LouisLetouzeyandThierryCoq•  Inspearit

–  (previouslyknownasDetNorskeVeritasFrance)

Page 64: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SQALE

Page 65: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Qualitymodel

Page 66: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Costs

Page 67: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Reducebusinessimapt

Page 68: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

MaximizeRoI

Page 69: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SonarQube

Page 70: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SonarQube

Page 71: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SonarQubeandSQALE

Page 72: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SQALE-likedashboardwithSonarQube

Page 73: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Structurallevel

•  Dependencyanalysis– PropagaJoncost

•  Interviewthedesigners– TDanditscausesareintheirheads

Page 74: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

DependencyStructureMatrix

A B C

AStrengthofB’sdependencyonA

BStrengthofA’sdependencyonB

StrengthofC’sdependencyonB

C

Page 75: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

DependenciesforMS-Lite

Page 76: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

DependencyStructureMatrix

Page 77: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

DependenciesinReleasePlanning

Dependencies between stories & supporting architectural elements

Understanding the dependencies between stories and architectural elements enables staged implementation of technical infrastructure in support of achieving stakeholder value.

Dependencies between architectural elements

Low-dependency architectures are a critical enabler for scaling-up agile development.1

Dependencies between stories

High-value stories may require the implementation of lower-value stories as precursors.2

1MaryandTomPoppendieck–“LeadingLeanSoUwareDevelopment”

2MarkDenne,JaneCleland-Huand–“SoUwarebyNumbers”

Page 78: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PropagaJoncost

•  “Density”oftheDSM– ProposedbyMcCormacketal.in2006– SeverallimitaJonsasatooltomeasureT.D.

•  ImprovedPC:– BooleantoconJnuousvalue(=dependency“strength”)

– Changesnotuniformlyspreadthroughoutthecode

– LesssensiJvetosizeofcodeMcCormacketal.2006

Page 79: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

SoTechnicaldebt…

•  …it’smessy

•  Cannotisolateortokenize– Lotsofdependencies

•  Cannotassesseasily– CostandvaluedependentonfutureevoluJon

•  Polymorphic– Good&bad,costlyandbeneficial,harmfulandinnocuous

Page 80: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PracJcalsteps

FromtacJcal(andsimple)tomorestrategic(andsophisJcated)

Page 81: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

•  TacJcal– Short-termacJons–limitedscope– Actualmeans:tools,processsteps,immiedateplan

•  Strategic– Long-termplan–widerscope– Process,management,educaJon– DrivesomeofthetacJcalacJonsabove

Page 82: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PracJcalsteps(1)-Awareness•  Organizealunch-and-learnwithyourteamtointroducetheconceptoftechnicaldebt.Illustrateitwithexamplesfromyourownprojects,ifpossible.

•  Createacategory“TechDebt”inyourissuetrackingsystem,disJnctfromdefects,ornewfeatures.PointatthespecificarJfactsinvolved.

•  Standardizeononesingleformof“Fixme”or“Fixmelater”commentinthesourcecodetomarkplacesthatshouldberevisedandimprovedlater.Theywillbeeasiertospotwithatool.

Page 83: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PracJcalsteps(2)-IdenJficaJon

•  AcquireanddeployinyourdevelopmentenvironmentastaJccodeanalysertodetectcode-level“codesmells”.(DonotpanicinfrontofthelargenumberofposiJvewarnings).

•  AUersome“triage”feedthemintheissuetrackingsystem,inthetechdebtcategory

•  Ateachdevelopmentcycle(iteraJon),reducesomeofthetechnicaldebtbyexplicitlybringingsometechdebtitemsintoyouriteraJonorsprintbacklog.

Page 84: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PracJcalsteps(3)-EvaluaJon•  ForidenJfiedtechdebtitems,givenotonlyesJmatesofthecostto“reimburse”themorrefactorthem(instaffeffort),butalsoesJmateofthecosttonotreimbursethem:howmuchitdragstheprogressnow.AtleastdescribequalitaJvelytheimpactonproducJvityorquality.Thiscanbeassistedbytoolsfromyourdevelopmentenvironment,tolookatcodechurn,andeffortspent.

•  PrioriJzetechnicaldebtitemstofixorrefactor,bydoingthemfirstinthepartsofyourcodethatarethemostacJvelymodified,leavingasideorforlaterthepartsthatarenevertouched.

Page 85: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PracJcalSteps(4)Architecturaldebt

•  RefineinyourissuetrackertheTechDebtcategoryinto2subcategories:simple,localized,code-leveldebt,andwideranging,structuralorarchitecturaldebt.

•  Acquireanddeployatoolthatwillgiveyouhintsaboutstructuralissuesinyourcode:dependencyanalysis

•  Organizesmall1-hourbrainstormingsessionsaroundthequesJon:“WhatdesigndecisiondidwemakeinthepastthatweregretnowbecauseitiscosJngusmuch?”or“Ifwehadtodoitagain,whatshouldhavewedone?”–  Thisisnotablamegame,orawhiningsession;justidenJfyhighlevelstructuralissues,thekeydesigndecisionsfromthepastthathaveturnedtotechnicaldebttoday.

Page 86: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

PracJcalsteps(5)–Processimprovements

•  Foryourmajorkindsoftechnicaldebt,idenJfytherootcause–schedulepressure,processorlackofprocess,peopleavailabilityorturnover,knowledgeorlackofknowledge,toolorlackoftool,changeofstrategyorobjecJves–andplanspecificacJonstoaddresstheserootcauses,ormiJgatetheireffect.

•  DevelopanapproachforsystemaJcregressiontesJng,sothatfixingtechnicaldebtitemsdoesnotrunyouintheriskofbreakingthecode.–  Counterthe“Itisnotreallybroken,soIwon’tfixit.”

•  IfyouareacJvelymanagingrisks,considerbringingsomemajortechdebtitemsinyourlistofrisks.

Page 87: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

References§  Brown,N.,Cai,Y.,Guo,Y.,Kazman,R.,Kim,M.,Kruchten,P.,etal.(2010).Managing

TechnicalDebtinSo)ware-IntensiveSystems.PaperpresentedattheFutureofsoUwareengineeringresearch(FoSER)workshop,partofFoundaJonsofSoUwareEngineering(FSE2010)conference.

§  Brown,N.,Nord,R.,Ozkaya,I.,Kruchten,P.,&Lim,E.(2011).HardChoice:Agameforbalancingstrategyforagility.Paperpresentedatthe24thIEEECSConferenceonSoUwareEngineeringEducaJonandTraining(CSEE&T2011),Honolulu,HI,USA.

§  Cunningham,W.(1992).TheWyCashPorLolioManagementSystem.PaperpresentedattheOOPSLA'92conference,ACM.Retrievedfromh^p://c2.com/doc/oopsla92.html

§  CurJs,B.,Sappidi,J.,&Szynkarski,A.(2012).EsJmaJngthePrincipalofanApplicaJon’sTechnicalDebt.IEEESoUware,29(6).

§  Denne,M.,&Cleland-Huang,J.(2004).So)warebyNumbers:Low-Risk,High-ReturnDevelopment,PrenJceHall.

§  Denne,M.,&Cleland-Huang,J.(2004).TheIncrementalFundingMethod:Data-DrivenSoUwareDevelopment,IEEESo)ware,21(3),39-47.

§  Fowler,M.(2009),Technicaldebtquadrant,Blogpostat:h^p://www.marJnfowler.com/bliki/TechnicalDebtQuadrant.html

§  Gat,I.(ed.).(2010).HowtoseSleyourtechnicaldebt--amanager'sguide.ArlingtonMass:Cu^erConsorJum.

§  Kruchten,Ph.(2010)ContextualizingAgileSoUwareDevelopment,”PaperpresentedattheEuroSPI2010conferenceinGrenoble,Sept.1-3,2010

Page 88: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

References§  Kruchten,P.,Nord,R.,&Ozkaya,I.(2012).Technicaldebt:frommetaphortotheoryandpracJce.

IEEESo)ware,29(6).§  Kruchten,P.,Nord,R.,Ozkaya,I.,&Visser,J.(2012).TechnicalDebtinSoUwareDevelopment:from

MetaphortoTheory--ReportontheThirdInternaJonalWorkshoponManagingTechnicalDebt,heldatICSE2012ACMSIGSOFTSo)wareEngineeringNotes,37(5).

§  Li,Z.,Madhavji,N.,Murtaza,S.,Gi^ens,M.,Miranskyy,A.,Godwin,D.,&Cialini,E.(2011).CharacterisJcsofmulJple-componentdefectsandarchitecturalhotspots:alargesystemcasestudy.EmpiricalSo)wareEngineering,16(5),667-702.doi:10.1007/s10664-011-9155-y

§  Lim,E.(2012).TechnicalDebt:WhatSo)warePracDDonersHavetoSay.(Master'sthesis),UniversityofBriJshColumbia,Vancouver,Canada.

§  Lim,E.,Taksande,N.,&Seaman,C.B.(2012).ABalancingAct:WhatSoUwarePracJJonersHavetoSayaboutTechnicalDebt.IEEESo)ware,29(6).

§  MacCormack,A.,Rusnak,J.,&Baldwin,C.Y.(2006).ExploringthestructureofcomplexsoUwaredesigns:Anempiricalstudyofopensourceandproprietarycode.ManagementScience,52(7),1015-1030.

§  Nord,R.,Ozkaya,I.,Kruchten,P.,&Gonzalez,M.(2012).Insearchofametricformanagingarchitecturaltechnicaldebt.PaperpresentedattheWorkingIEEE/IFIPConferenceonSo)wareArchitecture(WICSA2012),Helsinki,Finland.

§  McConnell,S.(2007)NotesonTechnicalDebt,Blogpostat:h^p://blogs.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx

§  SpecialissueofCuSerITJournalonTechnicalDebt,editedbyI.Gat(October2010)Cu^erITJournal,23(10).

§  Sterling,C.(2010)ManagingSo)wareDebt,Addison-Wesley.

Page 89: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

References(cont.)§  R.O.Spinola,N.Zazworka,A.Vetro,C.B.Seaman,andF.Shull,"InvesJgaJngTechnicalDebt

Folklore:SheddingSomeLightonTechnicalDebtOpinion,"inProceedingsofthe4thWorkshoponManagingTechnicalDebt,atICSE2013,P.Kruchten,I.Ozkaya,andR.Nord,Eds.,IEEE,2013.

§  K.Schmid,"OntheLimitsoftheTechnicalDebtMetaphor,"inProceedingsofthe4thWorkshoponManagingTechnicalDebt,atICSE2013,P.Kruchten,I.Ozkaya,andR.Nord,Eds.,IEEE,2013,pp.63-66.

§  K.Schmid,"AFormalApproachtoTechnicalDebtDecisionMaking,"inProceedingsoftheConferenceonQualityofSoUwareArchitectureQoSA'2013,Vancouver,2013,ACM.

§  Avgeriou,P.,Kruchten,P.,Ozkaya,I.,&Seaman,C.(eds)“ManagingTechnicalDebtinSoUwareEngineering(DagstuhlSeminar16162)”.DagstuhlReports(Vol.6,issue4pp.110-138).Dagstuhl,Germany:SchlossDagstuhl--Leibniz-ZentrumfürInformaJk.

Page 90: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Othersources(Talks/slides)•  Gat,I.,Heintz,J.(Aug.19,2010)Webinar:ReininginTechnicalDebt,

Cu^erConsorJum.•  McConnell,S.(October2011)Managingtechnicaldebt.(Webinar)•  Kniberg,H.(2008)Technicaldebt-Hownottoignoreit,atAgile2008

conference.•  Kruchten,P.(2009)Whatcolourisyourbacklog?AgileVancouver

Conference.h^p://philippe.kruchten.com/talks•  Sterling,C.(2009)h^p://www.slideshare.net/csterwa/managing-

soUware-debt-pnsqc-2009•  Short,G.(2009)h^p://www.slideshare.net/garyshort/technical-

debt-2985889•  West,D.(January2011),Balancingagilityandtechnicaldebt,Forrester&

CastSoUware

Page 91: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

Othersources•  SlidesonSonar,fromOlivierGaudin,CEOofSonarqube

•  SlidesonSQALEfromJean-LouisLetouzey,Inspearit

•  SlidesonDSMfromIpekOzkayaandRobertNord,SEI

Page 92: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg

ConceptualmodelofTechnicaldebt

Page 93: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg
Page 94: Strategic management of Technical Debt - Philippe  · PDF fileStrategic management of Technical Debt Philippe Kruchten April 4th, 2017 ICSA 2017, Göteborg