Quality Debt

Is your project going bankrupt?

© Planit Test Management Solutions Pty Ltd 2008

So . . . What is “Quality Debt”

Ward Cunningham: inventor of Wiki, and all around brainy guy

A Brief on “Technical Debt”

Shipping first time code is like going into

debt. A little debt speeds development so

long as it is paid back promptly with a rewrite.

. . . The danger occurs when the debt is not

repaid. Every minute spent on not-quite-right

code counts as interest on that debt. Entire

engineering organisation can be brought to a

stand-still under the debt load of an

unconsolidated implementation, object-

oriented or otherwise.-Ward Cunningham, 1992-

A Brief on “Technical Debt”

Debt can be simply broken into two parts:

• Principal:The direct cost of fixing those structural flaws in the production system – at a minimum is a cost calculated by:

• number of hours to fix

x

• fully burdened hourly cost of those involved – design, developing, testing

A Brief on “Technical Debt”

• Interest:Continuing costs: The amount of time (therefore money) developers spend in working around known design flaws, limitations, and bugs while coding new features – i.e. inefficiency

• See also: Business Risk, Liability, Risk, & Opportunity Cost

A “Quality Debt” Concept

• “Technical Debt” often does not present itself into defects during the testing or use of the system - It is a debt to the developers

• We (testers) deal with decisions that affect the quality of the system to the user - This is debt to the user / business . . . “Quality Debt”

Jordan Setters:Stylish Bass god, and the best tester in the world (according to him anyway)

A “Quality Debt” Concept

The first time code is released to production is like

going into debt. A little debt speeds the project

along, so long as it is paid back promptly with a

rewrite. . . . The danger occurs when the debt is not

repaid. Every minute the users spend on not-quite-

right functionality counts as interest on that debt.

Entire development projects can be brought to a

stand-still under the debt load of bad functionality,

defects or otherwise.

-Jordan Setters, 2012: shamelessly paraphrasing

Ward Cunningham -

What it looks like when we get it wrong

What it looks like when we get it wrong

• “Schools affected by the Xxxxxxx debacle are owed more than $16 million, according to a survey”

• “The actual costs could be as high as $20 million”

• The survey also showed 98% of principals have no confidence that Xxxxxxx will deliver an efficient payroll over the next year

What it looks like when we get it wrong

• “Thousands of teachers have been underpaid, overpaid or not paid at all, with consequences affecting tax returns, child support payments, ACC and superannuation”

• “A Ministerial Inquiry into the flawed system is expected to cost half a million dollars.”

TVNZ One News 11 February 2013

What it looks like when we get it wrong

What it looks like when we get it wrong

• No category 1 flags

• Most big systems will have some ‘warning bells’ attached to them – not unusual

• Can’t really look at them (defects) numerically

– Of such insignificance that they don’t really matter

– Actually . . . They weren’t as low ranking . . .

What it looks like when we get it wrong

• 137 software defects when system was ‘signed off’

• Assumed bugs would be ‘ironed out’ once live.

• In a report in March stated around 500 defects and 16000 “issues”

• Failed to evaluate the impact on the business and sector.

Types of Quality Debt

• Unintentional:

These are not strategic, but are a result of people doing a bad job and is often occurred unknowingly:– Bad Coding*

– A poor design approach

– Incomplete requirements*

Types of Quality Debt

• Intentional: Project makes a conscious decision to optimise for the present at the expense of the future. Characterised by decisions that include statements like:

– “If we don’t get this release done on time, there won’t be a next one”

– “We don’t have the time to . . .”

– “We can work around by . . .”

– “We’ll clean this up later”

– “We’ll do that after go live . . . in another phase”

Be sure you are taking on the right kind of Debt

• Some debt is taken on in large chunks:

“We don’t have time to do it right, so we’ll fix it after go-live” – akin to large debt like buying a car, it can be tracked and measured.

Be sure you are taking on the right kind of Debt• Other debt accumulates from taking hundreds of

short-cuts – akin to Credit Card debt

– Easy to incur unintentionally

– It adds up faster than you think

– It’s much harder to manage and track

Debt MUST BE SERVICED

• There is a tipping point

– Death by a thousand WTF’s

• The debt of the hundred little inefficiencies building up, adding time and frustration to their daily tasks

– This is “Interest” on your debt.

Often not paid by the project, but paid by the business in lost time and productivity.

Debt MUST BE SERVICED

When do you know you have Problem– A growing dislike for the system admitted by developers, testers, and

users

– Small defects never get fixed

– A bad user experience or UI is a clue it is rotten to the core

– Growing number of workarounds

– Consistently not meeting requirements

– Lack of knowledge of what the system does and why

– Unused features

Debt MUST BE SERVICED

When are you in over your head– Deep contempt for the system

– All knowledgeable developers, testers, and SME have disavowed the system or left

– Infrequent software releases

– Bug fixes take too long

– Lots of “Sorry”, “OMG”, “WTF”, and “$%#&*!!” comments

– Documentation is no longer useful as a test oracle

– Work arounds are now enshrined in the user experience

GMAS Example

Managing Debt in an Agile World

• The way debt can be measured and dealt with in Agile

• Hardening Iteration

• Debt a part of product backlog

• More opportunity to payback debt before it becomes an issue

• More likelihood of accumulating ‘credit card’ debt

How Do You Measure Quality Debt

• Maintain a “Debt List” with defects• Keep a track of the “Debt Backlog”

– when debt is paid off, remove it from the list; – if a debt starts becoming “old” then increase the priority of

that fix• Must have a useful measure to safeguard against the

accumulation of “credit card” debt

How Do You Measure Quality Debt

• You can calculate a cost for Quality Debt

(((0.1 LSV) + (0.25 MSV) + (0.5 HSV)) x Time(h) x $ /hour)

LSV = Low severity defect

MSV = Medium severity defect

HSV = High severity defect

• CAST surveyed 745 applications and the resulting analysis determined the average Technical Debt = $3.61 per line of code.

Communicating Quality Debt

• “For every (dollar) of competitive advantage gained by cutting quality, it costs $4 to restore it; and software is an organisational asset and decisions to cut quality must be made by executive management and reflected in the financial statements.”- Ken Schwaber

http://www.infog.com/presentations/agile-quality-canary-coalmine

Communicating Quality Debt

• This is a concept that immediately resonates with executives and project management

• Presents well to both non-technical and technical stake holders alike

Communicating Quality Debt

• Should be contextualised– Discuss in terms of money and time, rather than features

or functionality– If you can calculate the time of workarounds, then

compare with project business case– Compare “good debt” with “bad debt”

Communicating Quality Debt

• Treat the discussion as an on-going dialogue rather than a single presentation; it may take several discussions before the nuances of the metaphor fully sink in.

References

Curtis, Bill. Paying Down the Interest on Your Applications: A Guide to Measuring and Managing Technical Debt. New York: CAST, 2012

McConnel, Steve. Technical Debt. www.ontechnicaldebt.com, 2007

Schwaber, Ken. www.infog.com/presentations/agile-quality-canary-coalmine

Strutz, Nathan. Holistic Program Quality and Technical Debt (presentation). CF. Objective, 2011

Szynkarski, Alexandra. 3 Concepts to Aid Developers in Addressing Techinical Debt. www.ontechnicaldebt.com, 2012

http://www.ontechnicaldebt.com/http://www.infog.com/presentations/agile-quality-canary-coalminehttp://www.ontechnicaldebt.com/

Appendix A: Debt Glossary

• Technical Debt: These are the activities that a team or team members take shortcuts on now that will impede future development if let as is

• Quality (Testing) Debt: There is a diminishing ability to verify the functional and technical quality of software: the Break/Fix mentality

• Configuration Management Debt: Integration and release management become more risky, complex, and error-prone

• Design Debt: The cost of adding features is increasing toward the point where it is more than the cost of writing from scratch

• Platform Experience Debt: The availability and alignment of people to business objectives that involve software changes is becoming more limited or cost-prohibitive

• Non Debt: Feature backlog, deferred features, cut features, etc. Not all incomplete work is debt