Upload
oreilly-media
View
1.357
Download
1
Embed Size (px)
DESCRIPTION
Moneyball for Software Teams?Date: This event took place live on September 16 2011Presented by: Jonathan AlexanderDuration: Approximately 60 minutes.Cost: FreeThe new movie "Moneyball" starring Brad Pitt is about to be released. Based on the bestselling book of the same name, the movie explores the use of "sabermetrics" to build winning baseball teams. In this webcast, Jonathan Alexander, author of Codermetrics, suggests that these same ideas be applied to software teams. Jonathan will discuss how you can apply similar ideas to improve your software teams, giving examples of specific metrics and techniques to help you identify, analyze, and discuss the successes and failures of your software engineers and to help make the team a more cohesive and productive. If you manage or lead teams of software developers and engineers, and want to make sure that your focus is on success, you can't afford to miss this entertaining and instructive event.Presented by: Jonathan Alexander
Citation preview
Moneyball for Software Teams?
Presented by Jonathan Alexander VP Engineering at Vocalocity Author of Codermetrics (O’Reilly 2011)
The Popularity of Moneyball
Advanced stats used to analyze baseball players and teams Bill James, the father of sabermetrics, author and consultant Michael Lewis, author of Moneyball published 2003 Moneyball starring Brad Pitt being released Sept. 23rd, 2011 O’Reilly Strata Summit: The Business of Data (NYC Sept. 20-21) has Paul
DePodesta, VP NY Mets as a featured speaker about Moneyball
Metrics Have Changed the Game(s)
Scouting Drafting Trades Coaching Player Development Salary Arbitration
Principles of Moneyball:
Identify Undervalued Assets
Analyze the skills correlated with winning Discover important skills that market undervalues Leverage knowledge to build winning teams
© Brandon Vincent | Dreamstime.com
Principles of Moneyball:
Challenge Assumptions
Gather situational statistics Objectively check assumptions against facts Adjust strategy based on analysis and results
© Richard Kane | Dreamstime.com
© Eric Broder Van Dyke | Dreamstime.com
Principles of Moneyball:
Study and Learn from Outliers
anomaly (noun): an incongruity or inconsistency, a deviation from the norm
outlier (noun): a person or thing that lies outside, a point widely separated from the main cluster
Games At Bats Hits Doubles Triples HRs RBI Avg. OPS All Stars
Piazza ‘92-’07 1,912 6,911 2,127 344 8 427 1,335 .308 .922 12
Pudge ’91-’10 2,499 9,468 2,817 565 51 309 1,313 .298 .800 14
Techniques Used For Moneyball
Leverage basic performance statistics Hits, Runs Batted In (RBI), Runs Allowed (ERA)
Add “situational” statistics gathered by “spotters” Errors, out-of-zone fielding, pressure situations
Develop “advanced” statistics through combinations and formulas OPS (on-base plus slugging) , FIP (fielding independent pitching) ,
BABIP (batting average on balls in play) , WAR (wins above replacement)
Analyze statistics to find best predictors of individual and team success
Moneyball for Software Teams?
Implement new techniques to gather metrics on a wide variety of contributions
Find ways to measure “wins” and “losses” Analyze how individual contributions and team
“chemistry” correlate to wins and losses Examine Assumptions Discover Patterns
Use metrics to create focus and help identify opportunities to change, adjust, improve
The Magic Triangle Challenge
Oft-discussed “triangle”: Features-Time-Quality Is it true? You can’t add more work unless you
lengthen time or reduce quality
Avg. Complexity
Total Complexity
Quality Problems
Release Quality %
Release 1 1.2 272 86 68%
Release 2 1.6 248 77 69%
Release 3 1.5 274 109 60%
Release 4 2.8 318 69 78%
Release 5 2.4 347 88 75%
Release 6 1.4 261 92 65%?
Release Quality % = 100 – (Quality Problems / Total Complexity)
Questions To Answer
How well do team members handle their core responsibilities? Examples: Design, Code, Test
In what ways do team members contribute beyond their core responsibilities? Examples: Innovate, Take Initiative, Handle Adversity
How much do team members help others? Examples: Assist, Mentor, Motivate
Is the software team succeeding or failing? Examples: New Users, Production Bugs, Efficiency
What Are The Roles On Your Teams?
Playmakers and Scorers Defensive Stoppers Utility and Role Players Backups Motivators Veterans and Rookies
Example: Skill Metrics
Metric Description Formula
Points Measure the overall productivity of eachcoder on assigned tasks
Points = Sum of complexity rating for all completed tasks
Utility Measure how many assigned tasks eachcoder completes
Utility = Number of tasks completed
Assists Measure the amount of coder interruptionsand how much a coder helps others
Assists = Count of times that coder helps others
Saves Measure how often a coder helps fix urgentproduction issues
Saves = Number of severe product issues coder helps fix
Tackles Measure how many potential issuesa coder handles proactively
Tackles = Number of times a coder takes initiative or innovates
Turnovers Measure the complexity of assigned tasksthat a coder fails to complete
Turnovers = Sum of complexity for all tasks that are not completed
Errors Measure the magnitude of productionissues found in areas of coder responsibility
Errors = Sum of bug severity factored by population affected
Range Measure how many areas of software a coderworks on
Range = Number of areas worked on by a coder
Example: Response MetricsMetric Description Formula
Wins Measure the number of active users added Wins = Sum (User Activations)
Losses Measure the number of active users lost Losses = Sum (User Deactivations)
Win Rate Determine the average amount of time ittakes to get a “win” (new user)
Win Rate = Time elapsed divided by the number of new users
Loss Rate Determine the average amount of time ittakes to accumulate each “loss” (lost user)
Loss Rate = Time elapsed divided by the number of lost users
Win Percentage Measure the percentage of trials that successfullyconvert to active users
Win Percentage = (Successful Trials / Trials Completed) × 100
Gain Measure the number of Wins minus themissed opportunities and Losses
Gain = Wins - ((Trials Completed – Successful Trials) + Losses)
Penalties Per Win Measure the overall urgency of customersupport issues relative to the number of new users
Penalties Per Win = Penalties / Wins
Example: “Advanced” Metrics
Metric Description Formula
Power Measure the average complexity of the tasksthat a coder completes
Power = Points / Utility
Temperature Measure how “hot” or “cold” a coder is atany given time (start with Temp. 72)
Temperature = Previous Temp. × (Current Points / Previous Points)
O-Impact “Offensive Impact” to summarize how a coderhelps move projects along
O-Impact = Points + Utility + Assists
D-Impact “Defensive Impact” to summarize how a coderhelps solve issues or avoid larger problems
D-Impact = (Saves + Tackles) × Range
Plus-Minus Measure the amount of positive contributionsversus negative issues for each coder
Plus-Minus = Points - Turnovers - Errors
Teamwork Establish a relative rating for team-orientedcontributions
Teamwork = Assists + Saves + Range - Turnovers
Fielding Establish a relative rating for the range andbreadth of work successfully handled
Fielding = (Utility + Range) - (Turnovers + Errors)
Intensity Establish a relative rating for heightenedproductivity and dealing with hot issues
Intensity = Saves + Tackles + (Avg. Temp. – Start Temp.)
Win Shares Assign a relative level of credit to each coderfor new users
Win Shares = Wins × Influence × Efficiency
Loss Shares Assign a relative level of responsibility toeach coder for lost users
Loss Shares = Losses × (1.0 - Efficiency)
Techniques to Gather and Track Metrics Get data from existing systems
Project tracking, bug tracking, customer support
Instrument your software for usage data New users, lost users, feature usage, measured benefits
Self-reporting or “spotters” for situational data Create documents or database for metric storage and tracking
Increase Team Awareness of Skills
Create Focus on Team Results
Identify Key Goals and Accomplishments
Metric Description Formula
Boost Measure the amount of additional userbenefits delivered
Boost = Sum of the percentage of users receiving each benefit
Acceleration Measure the ratio of user benefits deliveredto urgent user issues created
Acceleration = Boost / Number of Urgent User Issues) x 100
Steps for Adopting Metrics
1. Find a Sponsor
2. Create a Focus Group
3. Conduct a Trial (restart or stop if trial fails)
4. Introduce Metrics to the Team
5. Create a Metrics Storage System
6. Establish a Forum for Discourse
7. Expand Metrics Used and Analysis
Places and Times to Use Metrics Regular Team Meetings (sprint retrospectives) Project Post-Mortems Mentoring Establishing Goals and Rewards Performance Reviews (validation) Promotion Consideration
Moneyball Strategies forBuilding Better Software Teams
Recruit for “Comps” Profile your team, identify roles you need, then recruit
Improve the Farm System Use interns, contract-to-perm, promote from within
Make Trades Re-organize teams internally to fill roles and balance skills
Coach the Skills You Need Focus on those with aptitude, use target metrics
Recruiting Comps
Defensive Stopper Profile Candidate A Profile Candidate B Profile Candidate C Profile
Avg. Points Medium High Medium Medium
Avg. Utility Medium Medium Medium Medium
Avg. Assists Medium Low High Medium
Avg. Errors Low Medium Low Medium
Avg. Saves High Low High Medium
Avg. Tackles High Low Medium Low
Avg. Range Medium Low Medium Medium
Best CandidateTarget Profile
Potentially Undervalued Assets
Defensive Stoppers Utility Players Backups
Resources for Further Exploration
Codermetrics: Analytics for Improving Software Teams
262 Pages
Released August, 2011
In bookstores, Safari Online, or at http://www.oreilly.com
Codermetrics.Org – community website
Post ideas or stories
Share resources (spreadsheets, analysis)
Ask questions
Post Events
Follow on Twitter @codermetrics
Special Offer
Visit http://oreilly.com to purchase your copy of Codermetrics and enter code 4CAST to save 40% off print book & 50% off ebook with special code 4CAST
Visit http://oreilly.com/webcasts to view upcoming webcasts and online events