Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
®
Copyright 2008 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 1
Inspiring, enabling and driving the Evolution of Quality at Adobe leveraging the TSP
Jim Sartain
Senior Director, Quality
September 22nd, 2009
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 2
Agenda
Background
Future Vision
How we inspire, enable and drive progress
Team Software Process at Adobe
Summary
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 33
About Adobe
More than $3 billion in revenue in FY2008, 7100+ employees
More than half of Adobe’s revenue is generated outside the United States
Industry leading technologies with global wide-spread adoption
Adobe® Reader® is distributed in 26 languages on 10 major platforms worldwide
Adobe® Flash® Player has 98% reach on connected PC’s and 8 million installs per day
More than 90% of creative professionals have Adobe Photoshop® software on their desktops
Major software development located centers in San Jose, San Francisco, Seattle, India, China, Germany and Romania
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 4
The Evolution of Quality at Adobe: 2012
Customers Employees Shareholders
70% of our customers are product promoters and detractors are rare
Shareholders see an increase in stock price tied to quality improvement savings
Love the fact they get twice as much and more important work done with less effort and drama
Aspects of this vision are already happening in some Adobe teams
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 5
The measureable piece of quality costs is the tip of the iceberg
Engineering Rework
Higher Support Costs
Refunds, Returns, Scrap, Patch downloads
For many software companies :
40% to 50% of Engineering Effort
50% of Customer Support Budget
Less time for innovation
Fewer promoters
Reduced product/service value
Delayed time to market
Increased employee attrition
Reduced employee engagement
Litigation costs
Indirect costs are typically two to three times direct
quality costs
Improving quality is a huge opportunity to free up resources for value-added work and make Adobe an even better place to work
Source: Principles of Quality Costs, Campanella
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 6
Increased Agility
Improved Customer Experience
Goals:
How we assess improvement progress
Improved Engineering Productivity
Progress on these goals is reviewed regularly by Senior Leadership
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 7
Increased Agility
Improved Customer Experience
Goals:
How we assess improvement progress
Improved Engineering Productivity
Are we delivering Customer Delight? Are key customer issues
being eliminated?
Questions:
How many days between code complete and release dates? Is effective Automated
Testing in place?
Are major sources of rework identified and reduced? Are teams using engineering
best practices to find and remove defects early
Progress on these goals is reviewed regularly by Senior Leadership
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 8
Increased Agility
Improved Customer Experience
Goals:
How we assess improvement progress
Improved Engineering Productivity
Are we delivering Customer Delight? Are key customer issues
being eliminated?
Questions:
How many days between code complete and release dates? Is effective Automated
Testing in place?
Are major sources of rework identified and reduced? Are teams using engineering
best practices to find and remove defects early
Progress on these goals is reviewed regularly by Senior Leadership
Metrics:
• Net Promoter Score• # support cases / customer
• % of dev effort spent on system testing and defect driven rework
• % defects found before System Testing
• # days effort per test cycle• Automated code coverage
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 9
Adobe(TSP teams)
50%
10%
>90%
Days
90%
Industry(Typical)
20%
50%
10%
Varies
Varies
What’s Important and How are We Doing?
Question
Improved Customer
Experience
ImprovedProductivity
IncreasedAgility
Goal
How satisfied are your customers? (Net Promoter Score)
% of dev effort spent testing/bug fixing?
% of defects found before system test?
Effort required to do a full test cycle?
Automated test code coverage level?
Sharing what’s possible and rewarding/recognizing improvement drives progress
WorldClass
70%
10%
>90%
90%
Hours
Sources: [Caper Jones , Applied Software Measurement,, 1996.; Software Assessments, Benchmarks, and Best Practices., 2000.
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 10
5
22
2
25 32
1405
10
0
100
200
300
400
500
600
700
Personal Review (Code)
Peer Review (Code)
Unit Test Integration Test System Test
Source: Six Adobe Projects using the Team Software Process
Minutes to Find and Resolve a Defect by Discovery Activity
TSP Projects are showing what is possible to the larger organization
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 11
Best Practice Adoption at Adobe
TSP
100%
75%
50%
25%
0%
% A
dopt
ion
Time From Introduction
Innovators Early Adopters Early Majority Late Majority Late Adopters
Source: Crossing the Chasm: Geoffrey Moore, 1991
Adoption of best practices follow a traditional technology adoption curve
Peer Reviews
Scrum
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 12
Peer Review Adoption at Adobe
60%30%
10%
How Likely are you to Recommend Peer Reviews to Others?
Promotors (9 or 10)
Neutral (7 or 8)
Detractors (0 - 6)
Source: 284 Survey Respondents
Enthusiasm for doing reviews is high
Jan Feb Mar Apr May June July Aug
Months of Effort Saved
Number of staff doing reviews is growing
More Peer Reviews are being done
Great progress… Adoption increasing while we improve effectiveness
0
500
1000
1500
2000
2500
3000
3500
Jan Feb Mar Apr May June July Aug
# Peer Reviews
Peer Reviews Managed Using Shared Code Review Server
Peer Review effectiveness is increasing
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 13
Scrum & TSP can work togetherSimilarities:
Incremental, Iterative, Empirical
Dedicated, self managing teams
Proven methods to deliver higher quality
Scrum:Light framework
Assumes teams will choose best eng practices
Low “startup costs”
Requires moderate changes for engineering
Good ramp up to high quality over time
TSP:More detailed methodology
Assumes teams will use PSP eng practices
Higher “startup costs”
Requires significant changes for engineering
Almost immediate leap to high quality
Combinations:
Scrum teams adopt TSP practices & metrics
TSP teams adopt scrum framework best practices
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 14
TSP Rollout Strategy
• Ensure organizational leadership expects and requires delivery of software products withquality
• Use TSP to establish examples of excellence• Only choose teams that are generally excited• Ensure each project has a strong coach• Use pilot teams as internal references and
sources of champions/future coaches• Build internal coaching capacity within each
adopting organization
A viral adoption approach works best
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 15
Team Software Process Adoption at Adobe
Since January 2008
Featured in keynotes at Adobe Tech Summits in San Jose, California and Noida, India
325 executives, managers, developers, and team members trained
Classes conducted in San Jose, San Francisco, Noida (India), Bangalore (India), Beijing (China), and Hamburg (Germany)
9 TSP projects underway, approximately 100 developers and quality engineers using TSP
Many teams are broadly adopting engineering practices learned from their TSP training
The rate of TSP adoption is accelerating
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 16162008 Adobe Systems Incorporated. All Rights Reserved.
Early Quality Results for our TSP Projects Are Impressive
Source: Quality Data for Six Adobe TSP Pilots
Adoption of TSP is a great way for teams to improve both quality and productivity
Total Cost of Quality = Quality Activities (e.g. Peer Reviews, Unit Testing) + Effort for Defect Rework (System Testing + Bug Fixing) TSP teams average of 9% is four times less rework than typical Teams found 93% of all defects before integration and system
test
0%20%40%60%80%
100%
Total Cost of Quality Post-Dev Yield (2)
TSP
Non-TSP
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 17
Other Benefits realized by teams adopting TSP
Teams are making plans they can fully commit to
Teams are taking ownership of quality
Decision making has been pushed to lowest possible level
Teams plan, track, measure, and improve
Measurement has encouraged other improvements
Improved design practices
Defect analysis to determine common causes and implement improvement methods
Better estimation methods
Automated unit tests (most teams are getting >80% statement coverage)
100% of designs and code inspected
The Benefits have been more than improved quality
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 18
What you must do to Inspire, Enable and Drive Improvement
Build an inspiring vision for the future
Ask senior leadership to require measurably better quality with each release
Use metrics to identify what is important and help the teams assess how they are doing
Be an enabler of teams… be a coach, not a cop
Broadly communicate the vision, plans and progress
Share successes and recognize improvement
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 19
Acknowledgements
Watts Humphrey for inventing the Team Software Process and his support in introducing it to Adobe
Noopur and Darryl Davis for the excellent training, consulting and coaching support that is helping make the initial projects successful
Johnny Loiacono and Barry Hills (Adobe) for their strong support of the Adobe Quality initiative
®
Copyright 2009 Adobe Systems Incorporated. All rights reserved. Adobe confidential. 20