Upload
glen-alleman
View
773
Download
0
Embed Size (px)
Citation preview
IntegratingAgileSoftwareDevelopment
withEarnedValueManagement
GlenB.AllemanPrimeProjectManagement+13032419633glen.alleman@niwotridge.com
IPMWorkshop 2015
2
Today’s Briefing
• How can Agile Software Development methods increase the Probability of Program Success (PoPS) on Earned Value programs?
• How can Agile development integrate with the FAR / DFAR and OMB mandates for program performance measures using Earned Value?
• What are the “touch” points (or possible collision points) between Agile and EIA-748-C?
• What are the measures of success for Agile methods in the context of EIA-748-C?
4
In case we’ve forgotten what EIA-748-C says
• EIA-748-C, page 1, defines the top level activities for a successful EV based project.
• We need to “connect the dots” between these and agile development.
§ Plan all work scope for the program from inception to completion.§ Break down the program work into finite pieces that can be assigned to a
responsible person or organization for control of technical, schedule, and cost objectives.
§ Integrate program work scope, schedule, cost objectives into a performance measurement baseline plan against with accomplishments may be measured. Control changes to the baseline.
§ Use actual costs incurred and recorded in accomplishing the work performed.§ Objectively assess accomplishments at the work performance level.§ Analyze significant variances from the plan, forecast impacts, and prepare and
estimate at completion based on performance to date and work to be performed.§ Use this information in the organization's management processes
5
When we Hear Agile, What Does That Mean?
† Dr. Ashton Carter, Under Secretary of Defense for Acquisition, Technology and Logistics, Sep/Oct, 2010 Defense AT&L
“Being able to turn inside the loop of unfolding events.”†
7
IMP Captures End User Requirements in Terms MOEs, MOPs, KPPs, and TPMs
SOWConOps
WBSCWBS
TechnicalPerformanceMeasures(TPM)
MessagesofPerformance(MOP)
MeasuresofEffectiveness(MOE)
KeyPerformanceParameters
(KPP)
IntegratedMasterPlan(IMP)
IntegratedMasterSchedule(IMS)
PerformanceMeasurementBaseline
(PMB)
TechnicalRequirements
8
Some Critical Concepts in Agile
Product Time
Capabilities
A high level system function completed across multiple releases
Features
Stories
A well define system function to be completed within a release
A small well defined system function that can be developed within one sprint
Release
Sprint
† Earned Value Management for Scrum Programs: Industry Best Practices, National Defense Industry Association Integrated Program Management Division (NDIA IPMD) Scrum EVM Working Group
9
Putting Earned Value Management and Agile Software Development Together
Feature 1,2,3Feature 4,5,6
Feature 7,8,9
Release 1 Release 2
Release 2 PP’s
WPPP
SLPPin IMS
CAEVM Reporting from Work Package Performance
EVM Performance• BCWS – from WBS
Basis of Estimate• ACWP – from time
cards• BCWP – from
completed Story Points
Sprints
Time Now
n Starting with the IMP, defined Capabilities are flowed to the PMBn Capabilities produce the value from each Releasen Control Accounts and Work Packages are on baseline in the PMBn Work Packages contain Features produced in each Release by Sprintsn Release Planning baseline for period of performance and BCWS
Vertical and horizontal integration of all work planned from the IMP/IMS down to Releases, Sprints, and Tasks in the Scrum Development Management system.
Performance Measurement Baseline
Scrum Software Development Lifecycle
Feature n
The Bright Line
Milestones
Data Items
IMP Program Events
IMP Accomplishments and Criteria
Scrum Development Control Account
Task
Task
Task
Task
Task
Task
Task
Task
Task
…
10
Ordinal Story Point cannot be basis of BCWS higher than a Feature
10
Feature 1
StoryStoryStoryStoryStoryStory
Team 1’s Uncalibrated Ordinal SP estimates
Feature n
StoryStoryStoryStoryStoryStory
Team 2’s Uncalibrated Ordinal SP estimates
∑ F1(SP) ∑ Fn(SP)
Release 1 ∑ of SP’s
• • •
§ At the Story level, relative effort defines individual estimates.
§ At the Feature level, lower level SP’s don’t have the same unit of measure in the way Dollars do.
§ When Features summed to the Release, relative measures do not provide basis of Physical Percent Complete.
Not same units of measure between Features – Uncalibrated SP’s
11
Building the PMB in a Agile Paradigm
WBS
Iteration 1 Iteration 2 Iteration 3 … Iteration n Close Out
§ Deliverables§ Tasks§ Tasks
§ Deliverables
§ Deliverables
§ Deliverables§ Tasks§ Tasks
CA CoDR … PDR
WBS basis of deliverables Backlog, per MIL-STD-881C, decomposed into Release Backlog, then into Iteration Backlog for delivery by Stories and Tasks.
13
CONNECTING THE MOVING PARTS INTO AN INTEGRATED WHOLE
With both Earned Value and Agile parts, let’s connect them into a Performance Measurement Baseline ready for execution and reporting in the IPMR
14
Three Systems are Needed for an Integrated Solution
• Shared data between each system with a single system of record.
• Planning flowed from IMS to Agile development system
• Physical Percent Complete flowed back to IMS then to EVMS
17
STEPS TO SETTING UP THE PMB FOR AGILE + EVM
Here’s the step-by-step activities to define the Performance Measurement Baseline (PMB) and get it on Baseline in Team Foundation Server, the IMS, and COBRA.
This includes Baseline Change Requests and statusing of the baseline to report EV in the IPMR.
This is the secret sauce of the principles and theory
18
Simple Rule for Earning Value in Agile
Each Iteration of each Release is a “value earning” opportunity
The next step is to connect Agile’s definition of Value with Earned Value’s
definition of Value
Business Value ≠ BCWP
19
Starting to “Connect the Dots” † ‡
Agile Point of View DoD Program Point of ViewRequirements evolve Scope agreed to and maintained
Simple designs are best Architecture thought out and maintained
Teams are self organizing Organizational structure establishes boundaries
Delivery teams establish best prescriptive processes High level guidance organizes work
Development teams know what to do
Process professionals define the boundaries
Agile teams work in an iterative manner
Product Development Lifecycle is serial over broader periods of time
† Abstracted from “Reality over Rhetoric,” Scott Ambler IBM Developer Works ‡ John Goodpastuer, Project Management the Agile Way
20
Let’s Start With 3 Simple Connections
Earned Value Management Agile
Both EV and Agile Development Measure Progress as Physical Percent Complete
+
1 Measures of progress in units of “physical percent complete.”
Each sprint produces 100% working product.
2 Forecast of future performance provided by past performance.
Forecast performance in units of product(s) produced.
3
A systems approach to the development of products and connecting Cost, Schedule, and Technical Performance.
Increasing fidelity of product and problem understanding takes place after each sprint and release.
21
EVM + Agile Process Flow
WBS
Story
Feature
PlaceWorkonBacklog
EstimateStory Points
WorkPackageBCWS
ExecuteSprint
RecordSPsComplete
CalculateP%CwithSPs
LoadP%CforWPinIMS
SendP%CtoCobra
CalculateBCWP
1. With the WBS decomposed to Features, Stories, and Story Point estimates
2. Place Features and Stories on Scrum backlog.
3. Perform work in Sprints.4. Record Stories earned and
calculate Physical Percent Complete (P%C)
5. Send to the IMS and to Cobra6. Calculate BCWP = BCWS ×
P%C in Cobra
The Bright Line
SOWConOps
22
Assess Performance On A Weekly Basis
Feature
Story
Story
Story
Story
Planned24SP
%Complete
100%
100%
0%
0%
Remaining4SP
Earned20SP
Week 1 Week 2 Week 3 Week 4
20 Day Sprints
Every Thursday Status§ Physical Percent Complete§ SP’s remaining to 100%
8 SP’s
8 SP’s
4 SP’s
4 SP’s
23
Here’s Those Connections
GL EVM Criteria Agile Approach1 Define WBS Features and Stories define tasks
2 Identify Organization Self organizing teams
5 Integrate WBS and OBS Self organized teams with a customer
6 Schedule Work Iterations and Releases
7 Identify Products & Milestones Working software at the end of iterations
8 Set time phased budget Fixed length iterations and releases
16 Record direct costs Fixed staff = Level of Effort
23 Determine variances EV + Velocity measures missed features
25 Sum data and variance Missed features moved to next iteration
26 Manage action plans Replan missed features, adjust velocity
28 Incorporate changes Replan missed features, adjust velocity
24
Provide managers with information at a
practical level of summarization
Relate time phased budgets to
specific contract tasks
Enable statistical estimation of
completion costs
Track and monitor
discrete project metrics
Communicate project status
Provide quantitative data for decision making
Provide a documented project performance trail
Alert project managers to potential schedule and cost risk
Program ControlsPractice
But First, Let’s Not Forget Business Management Practices …
…That Must Be Recognized Before Connecting Agile and EVM
25
11 EVM GL’s and 12 Agile Principles
Connecting Agile with Earned Value Management is actually obviousonce we get over the social aspects and focus on the Program Planning and Controls aspects. W
BS
OB
S
WB
S +
OB
S
IMS
Prod
ucts
and
Mile
ston
es
PMB
AC
WP
Varia
nces
Sum
of
Varia
nces
Cor
rect
ive
Act
ion
Rec
ord
Cha
nges
Early and Continuous Delivery ✔ ✔ ✔
Welcome Change ✔ ✔ ✔ ✔
Deliver working Software ✔ ✔ ✔
Business and DEV work together ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Motivated individuals ✔
Sustainable development ✔ ✔ ✔ ✔ ✔
Working SW measure of progress ✔ ✔
Face-to-Face communications ✔ ✔ ✔ ✔
Continuous technical excellence ✔ ✔
Maximize work not done ✔
Self-organizing teams ✔ ✔
Regular reflections ✔ ✔ ✔ ✔ ✔
26
Putting Earned Value Management and Agile Software Development Together one more time
Feature 1,2,3Feature 4,5,6
Feature 7,8,9
Release 1 Release 2
Release 2 PP’s
WPPP
SLPPin IMS
CAEVM Reporting from Work Package Performance
EVM Performance• BCWS – from WBS
Basis of Estimate• ACWP – from time
cards• BCWP – from
completed Story Points
Sprints
Time Now
n Starting with the IMP, defined Capabilities are flowed to the PMBn Capabilities produce the value from each Releasen Control Accounts and Work Packages are on baseline in the PMBn Work Packages contain Features produced in each Release by Sprintsn Release Planning baseline for period of performance and BCWS
Vertical and horizontal integration of all work planned from the IMP/IMS down to Releases, Sprints, and Tasks in the Scrum Development Management system.
Performance Measurement Baseline
Scrum Software Development Lifecycle
Feature n
The Bright Line
Milestones
Data Items
IMP Program Events
IMP Accomplishments and Criteria
Scrum Development Control Account
Task
Task
Task
Task
Task
Task
Task
Task
Task
…
27
The Reason EV + Agile is a Match Made in Heaven
The faster you obtain your intelligenceThe more intelligent your decisions
From an Aviation Week & Space Technology Ad†
Agile forces the measures of physical percent complete at the end of every Sprint.
By adopting this principle alone, EVM will have measures of Physical Percent Complete – Working
Software – assessment every two weeks
Otherwise you’re not doing Agile† L3 ad for UAV, AW&ST, September 14-27, 2014, pg. 37
29
PUTTING THESE IDEAS TO WORK
Using the Earned Value Management Intent Guide (EVMIG), here’s how to connect the dots at the next level down.
The 11 criteria of Earned Value connected with the 12 principles of Agile.
30
Assemble a credible WBS and Integrated Master Plan / Integrated Master Schedule (IMP/IMS)–WBS Dictionary says what will be built –IMP/IMS says how, where, when, and what will be built and the processes to build it.Identify Reducible – Event Based – uncertainties and the resulting risks from WBS Dictionary and IMP Narratives in the Risk RegisterPut these these risks in the Risk Register, with probability and impactsDevelop risk retirement plans and place them in the IMS using CBB funding for risk buy down.
30
Building a Risk Adjusted PMB in Steps
0
1
2
3
8
31
Assess the Irreducible – naturally occurring –uncertainties and the resulting schedule and cost risks in the WBS and IMS.
Use Monte Carlo Simulation to determine schedule margin and budget impacts from work activity durations to handle these irreducible uncertainties.
Assign schedule and cost margin to protect key item deliverables in the IMS.
31
Building a Risk Adjusted PMB in Steps
4
5
6
8
32
Determine cost and schedule impacts of unmitigated risks in the risk register and place them in Management Reserve, with their exposure and impact
Assemble mitigated Irreducible (Aleatory) and Reducible (Epistemic) risks with the unmitigated event–based risks into the Total Allocated Budget
– Risk handling an baseline for reducible and irreducible in the PMB.
– Risk Handling in Management Reserve.
32
Building a Risk Adjusted PMB in Steps (Concluded)
8
7
8
33
Steps to connecting the Agile Release Cycle with the Performance Measurement Baseline
1. Establish Releases
2. Establish Iterations within each release
3. Establish Stories from the WBS and allocate them to each release
4. Assign resources to each Story
5. Estimate work effort – in hours – to each story
6. Assess if capacity for work ≥ demand for work
7. Repeat Steps 4, 5, and 6 until demand equals capacity
34
GL 1: Define Authorized Work Elements
Define the authorized work elements for the program. A work breakdown structure(WBS), tailored for effective internal management control, is commonly used in thisprocess.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Work Breakdown Structure (WBS). § Road Map & Release Plan consisting of
Capabilities, Product Backlog & Iteration Backlog.
§ WBS dictionary (may or may not be used, but a method to reconcile the statement of work to the WBS structure must be demonstrated).
§ WBS dictionary: agile user stories are deliverables that you can measure “done” for, therefore user stories satisfy wbs dictionary.
35
GL 2: Identify Program Organizational Structure
Identify the program organizational structure, including the major subcontractorsresponsible for accomplishing the authorized work, and define the organizationalelements in which work will be planned and controlled.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Organization Breakdown Structure (OBS).
§ CAM just builds a team as usual, but the team needs to be persistent, and not interchangeable parts.
§ OBS intersections with the WBS.
§ Team hierarchy definition with resources associated with their sub–teams.
§ Done at the level of granularity to support the basis of estimate (BOE).
§ Persistent teams are needed to apply throughput benchmarks to product backlogs to validate plans.
36
GL 5: Integrate WBS and OBS
Provide for integration of the program work breakdown structure and the programorganizational structure in a manner that permits cost and schedule performancemeasurement by elements of either or both structures as needed.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Control accounts.
§ Evidence that the CA meets the 90% discrete work rule.
§ Defend schedule & cost performance at the CA level?
§ Agile CA = one release.§ Actuals captured at the story level.
§ Responsibility Assignment Matrix (RAM).§ Done at too high a level for the SW
development approach to make a difference.
§ Contract Performance Reports (CPRs), if applicable.
§ Given an objective of X stories in iteration Y, completed stories are earned; all unearned return to backlog and a new ETC is developed from the benchmarks & backlog.
37
GL 6: Schedule the Work
Schedule the authorized work in a manner which describes the sequence of work andidentifies significant task interdependencies required to meet the requirements of theprogram.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Integrated network schedules including master, intermediate (if any), and detailed schedules.
§ CAM’s agile roadmap becomes the auditable intermediate schedule demonstrating significant accomplishments (SA).
§ MRP or ERP schedules, or planned order reports.
§ Each task in IMS has associated resources.
§ Control account plans (may be separate plans or detail schedules).
§ CAM creates schedules compliant to DCMA 14 point assessment.
§ Work authorization documents. § Nothing different.
38
GL 7: Identify Products and Milestones
Identify physical products, milestones, technical performance goals, or other indicatorsthat will be used to measure progress.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Integrated schedules including master, intermediate (if any), and detailed schedules that identify contract milestones and key events.
§ Agile dev performance reporting follows the approved program system description
§ Apportioned technical performance milestones to reduce risk & roll up intermediate technical performance.
§ MRP or ERP production planned order reports. § Not relevant to sw development.
§ Control account plans (may be separate plans or detail schedules)
§ Not relevant to Software Development because we are reporting tasks as physical % complete, which will automatically roll up.
39
GL 8: Set Time Phased Budget
Establish and maintain a time–phased budget baseline, at the control account level,against which program performance can be measured. Initial budgets established forperformance measurement will be based on either internal management goals or theexternal customer negotiated target cost including estimates for authorized butundefinitized work.EVMIG Objective Evidence Agile Objective Evidence for EV
§ Control account plans. § Time phased budget created for the current iteration(s) and future work.
§ Summary level planning packages.
§ Agile summary level planning documented in road map. Comprises capabilities, features and stories
§ Agile planning packages driven by persistent teams with proven benchmarks.
§ Performance Measurement baseline.
§ Is there a target threshold for future work as described in a PMB? Within 10% OTB?
§ Undistributed budget logs. § Does this have anything to do with SW dev approach?§ Notification to the customer
of an over–target baseline. § Does this have anything to do with SW dev approach?
§ Work authorization document. § Does this have anything to do with sw dev approach?
40
GL 16: Record Direct Costs
Record direct costs in a manner consistent with the budgets in a formal systemcontrolled by the general books of account.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Reconciliation of project costs with the accounting system.
§ CAM would follow program direction on these.
§ These are not impacted by sw dev approach
§ Actual costs are reported at the control account level at a minimum.
§ Not impacted by SW development approach.
§ Reconciliation of subcontract reported actual costs to subcontract payments.
§ Not impacted by SW development approach.
§ Internal and external performance reports for subcontractors.
§ Not impacted by SW development approach.
§ Subcontractor control account plans, when utilized.
§ Not impacted by SW development approach.
41
GL 23: Determine Variances
Identify, at least monthly, the significant differences between both planned and actualschedule performance and planned and actual cost performance, and provide thereasons for the variances in the detail needed by program management.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Variance analyses (budget based schedule variances and cost variances).
§ Can track & report variances per the approved program system description
§ Management action plans. § Actionable recovery plans per issue.
§ Updated schedule task completion and cost–at–completion forecasts.
§ Scrum Agile has a POD and Plan for Iteration.
§ CAM’s monthly EAC reporting follows the approved program system description
§ Project schedules and schedule analysis outputs.
§ PM tracks the dynamic backlog, which will go up and down based on sponsor feedback
42
GL 25: Summarize Variances
Summarize the data elements and associated variances through the programorganization and/or work breakdown structure to support management needs and anycustomer reporting specified in the project.
EVMIG Objective Evidence Agile Objective Evidence for EV§ Variance analyses. § There is nothing in Agile’s approach to SW
development that precludes reporting variances at the WP level.
§ Agile is more dynamic than EVM so variances are less the issue than the evolving baseline, as approved in governance. The sponsor will want to track accumulating business value and variances to total product needs.
§ Schedule and cost performance reports.
§ Similar – but measures of performance not usually in dollars
§ Management action plans. § Similar – but less formal. Collaborative discussion of what actions to take include the customer.
§ Updated schedule and cost forecasts.
§ Similar – but less formal. Planning processes include the customer.
43
GL 26: Implement Management Plan
Implement managerial action taken as the result of earned value information.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ To–Complete Performance Index (TCPI).
§ TCPI = Work Remaining / Cost Remaining ((BAC –BCWPcum) / (EAC – ACWPcum)). In Agile, work remaining is the product backlog. Backlog is BAC – BCWP.
§ Independent completion estimates. § No longer used in 2010
§ Risk management data and similar metrics.
§ Qualitative Risk Burn–down Chart (risk rating)
§ Management action plans and review briefings.
§ Agile approach called Commitment Based Planning –where the SCRUM team makes and meets its time phase BCWS commitments.
§ Any team, when behind, gives voice to the customer when evaluating/reweighting the triple constraint.
§ Variance analyses. § This is an issue of cost mgmt and system description would define when and where team members would bill
44
GL 28: Incorporate Changes (1)
Incorporate authorized changes in a timely manner, recording the effects of suchchanges in the budgets and schedules.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Contractual change documents.
§ Bug reports, new user stories, but not necessarily cost sized.
§ User stories above baseline are tracked as new scope (with a valid BOE) and require BCWS
§ Change control logs (management reserve, undistributed budget, performance measurement baseline, and contract budget base).
§ New or materially altered features or stories are changes.
§ Control account/work package/planning package plans.
§ Product and iteration backlogs are frozen during the development period
45
GL 28: Incorporate Changes (2)
Incorporate authorized changes in a timely manner, recording the effects of suchchanges in the budgets and schedules.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Master schedules, intermediate schedules (if any), and detailed schedules.
§ Iterations and evolutionary planning at the detailed levels merges with the end to end planning for agile.
§ Statement of work, WBS, and WBS dictionary.
§ Customer owner and Planning processes identify requires work and its description.
§ Work authorization documents. § Planning sessions, authorize a set of Stories to be developed during the iteration.
§ Management reports (contract performance reports or other applicable management reports).
§ Big Visible Charts, “sticky notes” display progress to plan for the agile team.