Failing Fast - An Autopsy of a Failed Release

Agile Cambridge 2015, John Shaw

CASE STUDY Failing Fast - An Autopsy of a Failed Release

Background

3 Copyright © Capgemini 2015. All Rights Reserved

Failing Fast – Agile Cambridge 2015 | October 2015

Background

Government Organisation

Funding Administration Application

Start of Beta



Timeline – Start of Beta Phase

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



Technologies

(IaaS v PaaS)

(private v IaaS v SaaS)



Delivery Team

Programme and Project

Management

Architecture • Lead Architect

• 2 x Architects

Product Team • Product Owner

• 3 x Product Specialists

• 3 x Business Analysts

• 1 x UX Specialist

Development • Delivery Manager

• 2 x Scrum Masters

• 1 x .NET Build Engineer

• 3 x CRM Developers

• 12 x C#/.NET Developers



Release Planning

Self-organised three teams

Four two-week sprints



Sprint 1

Goal: product selection



Sprint 1



Sprint 1

Outcome: partial success



Sprint 2

Goal: ESB integration



Sprint 2



Sprint 2




Sprint 3

Goal: ESB deployment



Sprint 3



Sprint 3

Outcome: failure



Failure!

Immediate Action:

Inspect the Velocity



Sprint 4

Goal 1: evaluate .NET ESB

Goal 2: ESB deployment



Sprint 4



Sprint 4


Causes



Root Cause Analysis – Velocity

ESB technology poor fit for the dev team

WHY? Dev team unfamiliar with Java technologies

WHY? Developers mix of CRM and C#/.NET

WHY? That’s what was requested for procurement

WHY? The organisation have an established Microsoft capability



Root Cause Analysis – Deployment

Difficulty deploying Java technology

WHY? Build pipeline was based on Microsoft technologies

WHY? Specialist build engineer was Microsoft stack only

WHY? Java build pipeline would never get the attention

WHY? Based on Microsoft TFS

WHY? Based on Microsoft Powershell

WHY? Based on Microsoft Azure

WHY? Restricted to IaaS (no PaaS tooling)



Mixed Stack of Technologies

(IaaS v PaaS)



Timeline – Alpha Phase

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



Why?

Procurement decision during alpha

Package selection during alpha



Package Selection

Jboss Fuse

selected for good operational reasons

Excellent operational dashboard

Configuration over code

Re-planning



Timeline – Re-launch of Beta

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



Options to Proceed

Replace some or all of the dev team



Options to Proceed

Train the existing team



Options to Proceed

Change the ESB technology



Decision to Proceed

Backing from the board



Decision to Proceed

Increase in Velocity?



Decision to Proceed

Business Risk



Timeline – Re-launch of Beta

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



Sprint 1



Re-launch

Impact on Estimation



Re-launch

Product Backlog



Re-launch

Operational Considerations



Sprint 4



Timeline – Go-live (1st Release)

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



1st Release

Delivered!

Business milestone met!

Continuous

Improvement?



Continuous Improvement?

Pressure to Deliver



Continuous Improvement?

Tactical Choices

Success!



1st Release - Success

“We are delighted that the first Release of [the system] was used ... [to process] about 50,000 individual field updates in 56 minutes - an

enormous improvement when compared to the 3 days it would normally take to import through [the previous system]. All the reports that we’ve had from [the business] say that they are impressed with

the system.”



Timeline – Go-live (2nd Release)

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



2nd Release - Success

“It paid 503 [recipients] with 1,579 transactions a total of £217 million.”



Timeline – Go-live (3rd Release)

2013 2014 2015

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Febru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er

Ja

nu

ary

Fe

bru

ary

Ma

rch

April

Ma

y

Ju

ne

Ju

ly

Au

gu

st

Se

pte

mb

er

Octo

be

r

No

ve

mb

er

De

ce

mb

er



3rd Release - Success

“The payment files were successfully processed from [the service] to [the finance system].”

Reflections



Reflections



Reflections

Technical Debt



Reflections

Empirical Process

www.capgemini.com

The information contained in this presentation is proprietary.

© 2015 Capgemini. All rights reserved. Rightshore® is a trademark belonging to Capgemini.

About Capgemini

With almost 145,000 people in over 40 countries, Capgemini is

one of the world's foremost providers of consulting, technology

and outsourcing services. The Group reported 2014 global

revenues of EUR 10.573 billion.

Together with its clients, Capgemini creates and delivers

business and technology solutions that fit their needs and drive

the results they want. A deeply multicultural organization,

Capgemini has developed its own way of working, the

Collaborative Business ExperienceTM, and draws on Rightshore ®,

its worldwide delivery model.

We’re hiring! http://www.uk.capgemini.com/careers

[email protected]

@johnfcshaw

http://www.facebook.com/Capgemini

http://www.linkedin.com/company/capgemini

http://www.twitter.com/capgemini

http://www.youtube.com/capgemini

http://www.slideshare.net/capgemini

Software

Failing Fast - An Autopsy of a Failed Release