Capability Systems Centre

capabilitysystems.unsw.adfa.edu.au

Designing Capability Through Research

Crafting and Implementing Project Execution Strategies

Crafting and Implementing Project Execution Strategies for Complex

Projects

John Davies

DISC Seminar – 12 November 2017

BackgroundDealing with complexity is a key challenge with system acquisition and sustainment becoming more interdependent with other capabilities (existing and emerging), rapid technological advancement, and greater emergence associated with end-user needs. This seminar explores the dimensions of complexity and offers guidance on how appropriate project execution strategies can best deal with this complexity.

Seminar Outline

0900-1200• Introduction (complexity and the failure of business as usual

approaches)• Risk Management• Selecting and Implementing Project Execution Strategies1245-1600• Governance• Risk Management in Practice• Project Execution Strategies in Depth

What is Complexity

‘Complex’ programs are not necessarily ‘complicated’.The United Kingdom National Audit Office (UK NAO) defines complexity as follows:• at the outset there is uncertainty over the route to delivering

the project outcome, or the project has aspects that have not previously been encountered; or

• there is a high level of change in the outcome required during the project’s lifetime

Keep in mind that ‘change’ could be simply a symptom of sloppy project management and systems engineering practices.

Dimensions of Complexity

Remington and Pollack

Assessment of ComplexityTreating programs as a binary construct of either ‘complex’ or ‘simple’ is a dangerous proposition. Some elements of a program may be simple whilst others can be quite complex. Similarly, a project can start out complex and become simple (or vice versa).

Typical means of defining complexity

• The Kraljic matrix• Acquisition Category (ACAT)• ICCPM (Complexity Assessment Tool)• Helmsman (Project Complexity Framework)• Canadian Government http://www.tbs-sct.gc.ca/hgw-

cgf/oversight-surveillance/itpm-itgp/pm-gp/doc/pcra-ecrp-eng.asp

Helmsman complexity assessment -A ‘Comparison of Project Complexity between Defence and other Sectors’

CASG Risk and Complexity

• Smart Buyer Framework– Core Acquisition and Core Sustainment Risks Drive

the Project Execution Strategy– No hard rules (unlike the previous Acquisition

Category (ACAT))

Smart Buyer – Detailed Design (2016)Core Acquisition Risk Categories Core Sustainment Risk Categories

Requirements In-Service Requirements

Technology Obsolescence

Schedule Commercial

Commercial Fundamental Inputs to Capability

Project Integration Financial

Defence Integration Strategic

Financial Operational

Strategic Industry

Industry

Problems with Complexity Assessment

• Uncertainty– Epistemic– Ontological

“We don’t know what we don’t know”

Further Problems with Complexity Assessments

• Temporal dimensions– Risks emerge and retire throughout the

acquisition and sustainment life cycle• Structural issues

– Applying a complexity ‘grade’ to the whole capability lifecycle and all sub-systems

Systems and Processes

• Many organisations seek to classify procurement into discrete categories.– Is this consistent with a ‘smart buyer’?– Do the cost savings associated with templates,

training and procedures outweigh the loss of flexibility and tailored approaches?

The Coastline Paradox

Refer Andrew Pyke Presentation PCSG 2016

Source: hLps://41.media.tumblr.com/cdb7b518ed96f5d1ba3abace99077638/tumblr_n5s9nsUsw1sutkk1o1_500.png

Arnold Rimmer’s Study Plan“The first week of study, he would always devote to the construction of a revisiontimetable. Weeks of patient effort would be spent planning, designing and creating a revision schedule which, when finished, were minor works of art.

Every hour of every day was subdivided into different study periods, each labelled in his lovely, tiny copperplate hand; then painted over in watercolours, a different colour for each subject, the colours gradually becoming bolder and more urgent shades as the exam time approached. The effect was as if a myriad tiny rainbows had splintered and sprinkled across the poster-sized sheet of creamwove card.

The only problem was this: because the timetables often took seven or eight weeks, and sometimes more, to complete, by the time Rimmer had finished them the exam was almost on him. He'd then have to cram three months of astronavigation revision into a single week. Gripped by an almost deranging panic, he'd then decide to sacrifice the first two days of that final week to the making of another timetable. This time for someone who had to pack three months of revision into five days.” Grant Naylor, Red Dwarf – Infinity Welcomes Careful Drivers (1989)

Goldilocks Planning?• Not too much, not too little!• Ask yourself: what is the plan for?

– To Seek funding and create budgets– Identify resources– Achieve gate review milestones– Demonstrate confidence to stakeholders

Second Order Learning

“If the only change that can be contemplated takes place in the context of an existing mental model, then you are limited to bringing about first-order learning. If, however, the mental model itself can be changed, and purposes radically altered, then second-order change is possible.”

- M. Jackson, “Systems Thinking” (2003) p10.

Key Challenges

• Striking the balance between consistency and responsiveness– Resources– Repeatability– Centralised, Decentralised and Centre-led

approaches– Risk

Business as usual?

• Linearity (water fall development life cycle)• Too much planning (reductionist approaches)• Adoption of worst practice?

Key Success factors for Complex Projects

ICCPM ‘Contracting for Success in Complex Projects’ (2015)1. Clearly defined project goals and vision,2. Appropriate Relationship/behavioural management,3. Prudent risk management and equitable risk allocation,4. An acquisition and sustainment strategy suited to the project

at hand,5. Leadership and competencies of the team, and6. A robust project management and systems engineering

framework (hygiene factors)

Traditional Approach for Goal Alignment

The principal defines outcomes in precise terms (statements of requirement and statements of work) with a robust contractual framework that precisely allocates liability.

Defence Industry

LinearityGate 0 –> Gate 1 –> Gate 2The Capability Lifecycle Manual now contemplates skipping Gate 1 for non-complex projects and direct source options and having multiple Gate 1 reviews for the most complex projects.

Gate Reviews“Gate 2 - Assure decision makers the proposal to commit public money is sound and should proceed with the acquisition”.

- The Capability Life Cycle Awareness Presentation (2016)

What does this mean? (acceptable risk, requirements stability?)

Post Gate 2

“In the event contractual terms within that envelope cannot be agreed, the Project will return to the Investment Committee with a recommendation on whether to terminate the Project or expand the performance/time/cost envelope.” Capability Life Cycle Manual (2016)

Traditional Approach to Managing Relationships

• A focus on liability assignment• liquidated damages• termination for convenience• no waiver clauses• onerous contract change proposal/variation

processes• Some attempts for Alternate Dispute Resolution

(ADR)

Traditional Approach for Risk Allocation

Abrahamson’s Principles of transferring the risk to the party best able to manage the risk if often ignored. Consider the following:• The principal assumes no responsibility for the reliability of

the tender documentation,• The tenderer must acquaint itself with all information, and• Information is provided for the convenience of tenders only

and this information is not guaranteed. J. Feehley ‘Tendering and Contract Documentation’ (1998)

The Illusion of Risk Transfer

• Contracts cannot legislate for every permutation of risk

• Rarely will parties have ‘clean hands’• Risk take back is often the norm in complex

projects

Contracts – not a panacea• The role of the lawyer is to legislate for risk in

accordance with management’s instructions.• Problems

– GFE/GFI/GFD/GFF – baseline risks– Interdependencies with other projects or contracts– Uncertainty– Variations (there is rarely price competition with

variations)– Interpretation (fitness for purpose)– Insurance– Implied terms

Clean Hands and Risk Take Back

• IPT participation• Risks outside of the contractor’s control• Actions and behaviours modify the contract• Poor Contract Configuration Management

“A revised schedule is to business what a new season is to an athlete or a new canvas to an artist.”

N. Augustine ‘Augustine's Laws’ (1986)

Inappropriate Project Execution Strategies

One size will rarely fit all yet organisations love ‘boilerplate’ templates! Failure to vary the remuneration strategy as a function of lifecycle and risk.

Lack of Leadership

Leadership is a key component for supporting collaborative behaviours as observed by the UK NAO:“Every case study ranked leadership as the most important factor in developing collaborative relationships.”

UK NAO Good Governance ‘Measuring Success Through Collaborative Working Relationships’ (2006)

Worst Practice Leadership"Managing the NHS IT suppliers is like running a team of huskies. When one of the dogs goes lame, it is shot. It is then chopped up and fed to the other dogs. The survivors work harder, not only because they have had a meal, but also because they have seen what will happen should they themselves go lame." UK Parliament, Committee of Public Accounts (26 June 2006)

Don’t abandon the Hygiene factorsA final success factor for complex projects is to ensure all the procurement ‘hygiene’ factors are supported. Adopting all the other attributes will be meaningless if we do not have competencies in: • Project Management• Configuration Management• Quality Assurance• Systems Engineering• Commercial and Contracting Disciplines• Logistics

Risk Management and Complex Projects

• The acquisition and sustainment strategy is crafted to deal with known risks.

• Risks are typically high in complex projects• We also need to deal with uncertainty and

flexibility

Reminder - Complex Project Risk Management Challenges

• Very long implementation timeframes• Multiple Parties• Diverse stakeholders• Difficulty in assigning responsibility• First of type

Industry Risks

• Cash Flow• Demand/usage• Financing• Indices/fluctuations• Risk adjusted rates of return• Solvency• Insurance

Contract Risks

• Jurisdictional• Legislation change (Tax, R&D credits, Industrial

Relations, WH&S, Contract Administrative Law)

• Proportionate liability

What are the processes for identifying risk

• Customer Drivers, Risks, Assumptions and Issues Log

• Liability Risk Assessments– Craft Project Execution Strategy to manage these

known risks• What are the problems with this approach?

How do we traditionally deal with risk

• Accept the risk• Treat the risk• Avoid the risk• Transfer the risk (or partial transfer with risk

sharing)

How does a contract deal with risk?

• Remuneration Strategy • Warranties• Limitation of liability• Indemnities• Liquidated damages• Consequential damages• Breach/remedies

Problems with risk management approaches in complex projects

• Clinical risk transfer is often not possible• Multi party involvement exacerbates problems

with allocation of risks• Failure to anticipate uncertainty• Failure to consider temporal dimensions of

risk (see example)

Time

Risk

Complex Project Risk Management Best Practice

• Early Industry Involvement • Offer Definition and Improvement Activities• Risk Sharing where appropriate• Joint Decision Making• Joint Risk management• Collaboration• Agile contracts

Early Industry Involvement

• The quality of risk identification and classification will improve with early industry involvement

• Probity Issues need to be carefully managed• The use of Early Contractor Involvement

Strategies improve the quality of the risk management function.

Risk Sharing

• Parties are more likely to be proactive in risk management if they have ‘skin in the game’.

• Options for risk sharing include– Remuneration (target cost, gainshare/painshare)– Achievement of Key Performance Indices– Shared costs (e.g. insurance)

Joint Decision Making

• Joint decision making does not mean that the customer’s decision making ability is fettered, rather joint decision making allows for:– Collaborative approaches to dealing with change– Support for agility– Dealing with emergence

Joint Risk Management

• Complex projects often fail because they do not take a holistic view of risk.

• Joint risk management may involve joint risk logs to avoid duplication and errors of omission.

Collaborative Contracts

• No hard rules – there is always a spectrum of Collaboration

• Refer CASG Collaborative Contracting Better Practice Guide (2017)

Agile Contracts

• No contract can anticipate every outcome.• Agile contracts allow for off-ramps, rescoping,

and re-negotiation• We need to understand industry’s need for

certainty and ensure arrangements are both fair and contestable.

No Such Thing as a Free Lunch

• Risk Sharing will attract new risks in itself• Costs of bespoke contracts• Behavioural alignment• Trust!

Risk Management Pitfalls

• Optimism Bias• Deviant Behaviours• Governance and Legislation

Megaprojects and Risk

• Rail – 45% cost overruns• Tunnels and bridges – 34% cost overruns• Roads – 20% cost overruns

Flyvbjerg et al (2003)

Weapon System Cost growth averages 46% RAND Corporation ‘Sources of Weapon System Cost growth’ (2008)

RAND Corporation ‘Sources of Weapon System Cost growth’ (2008)

Optimism Bias - causes

• Strategic misrepresentation• Adversity to accept sunk costs• Cost Benefit Analysis shortcomings• Lack of commercial focus

Potential Deviant Behaviours

• Placing too much of your supplier’s profit at risk

• Inappropriate KPIs • Too many KPIs• Creating monopolies• Asymmetry of information (what is a smart

customer?)

You get what you measure!

“In one (perhaps legendary) case, a key performance measure was the time taken to answer phone calls. The target was met by the contractor simply picking up the phone and hanging up again.”

ANAO ‘Developing and Managing Contracts - Better Practice guide’ (2012)

Contracting and Procurement Options

• What are business as usual approaches?• Why do they often fail in complex projects?• What can we do to improve upon the status

quo?

Background

The terms procurement option, procurement strategy, and acquisition and sustainment strategy are often used synonymously to describe:• Contract method• Market engagement method• Finance approach• Ownership strategyAcross the whole system lifecycle

Construction & Engineering ModelFocus on a two stage process:a. Designb. Constructione.g. AS 4300-1995 General Conditions of Contract for

Design and ConstructThis does not cater for other procurement options such as services contracts, R&D, spiral development, evolutionary acquisition, IT etc.

Procurement Option NomenclatureConventional Procurement Options• Traditional: Construct only contracts; and• Non-traditional:

i. Design and Construct (D&C) (or Design and Implement),ii. Design Construct and Maintain,iii. Managing Contractor/Contract Management,iv. Early Contractor Involvement, (ECI), and v. Spiral development or rapid prototypes (for software programs).

Non-Conventional Procurement Options• Alliance Contracts• Public-Private Partnerships

Traditional Contracts (Fixed Price Construct Only)Benefits

– Minimal Resources needed from the Buyer – Price competition demonstrates value for money (contra cost

reimbursement)– Quantitative emphasis on tender selections– Retention of legal remedies– Visibility of costs and schedule before contract signature– Adoption of Abrahamson’s principles of risk allocation – Mature insurance products– Arms length contract

Traditional Contracts (continued)Costs

– Buyer liable for errors/omissions in the design– Greater likelihood of variations/contract change

proposals– Greater risk of no bids/inflated bids– Little or no scope for innovation – Greater likelihood of disputes

- Buyer must retain some design expertise

Design and Construct ContractsBenefits

– Same as traditional but buyer is no longer liable for design errors/omissions.

– Lower resources required from principal (no need to retain in house design expertise)

– Greater scope for innovationCosts

– Same as traditional but buyer loses control over the design- May introduce IP issues when contractor adopts proprietary design solutions.

Project ManagementDelegation of Project Management Responsibility (EPCM/PCM)• e.g. AS4916-2002 and AS4915-2002

There is a clear distinction between administrative versus contractual obligations

Defence examples – Integrated Support Contracts (above the line), Major Service Provider

R.Quick “QLDs ECI Contract”(2007)

Benefits– Same as D&C– Less management resources needed from buyer

Costs– buyer loses control over management– Insurance becomes more complex

Managing Contractor

A form of outsourcing where the customer appoints a Managing Contractor (MC). The MC contracts directly with suppliers.The MC does not assume all risks associated with suppliers.

Benefits– Less resources required from Customer– MC incentivised to achieve superior outcomes

Costs– May be a limited pool of suppliers– Loss of Control

Early Contractor InvolvementFirst StageCost reimbursement for the initial design and risk reduction activities. Develop target cost and schedule for Stage 2.Second StageIncorporates Construction/Implementation (typically guaranteed maximum price)

Benefits– Allows for early identification of risks.– Best suited for equitable risk allocation and the delivery of

value for money– Greater scope for innovation– Attractive to industry

Costs– Requires substantial buyer resources in the first stage– May be difficult to introduce price competition into tender

selection.– Not suitable where substantial risks exist in the construction

stage (stage 2)

Discussion

• How does the ECI process compare to Offer Definition and Improvement Activities?– Multi stage tender process– Early identification of potential suppliers– Less reliance on adversarial negotiations

Project Alliances• Risk Sharing• No disputes• No Blame• Open Book financial reporting• Best for Project Principles• Express Good Faith Provisions

Refer ‘National Alliance Contracting Policy and Guidelines’ (2014)

Alliance Variations• Pure Alliance (no price competition)• Multiple TOC alliance (price competition)• Hybrid Alliance (carve out risk sharing/joint project teams)

Alliance Benefits• Creation of a commercial framework which aligns the

interests of all parties;• Improved risk management especially with uncertain

project requirements and environments;• Earlier involvement in preliminary design activities

providing greater visibility of project costs;• Reductions in resources needed to administer contracts,

especially contract change proposals;• Improved project performance and innovation;• Greater transparency in project prices; and• Attractive to industry (higher likelihood of bidding).

Alliance Costs- An absence of legal recourse should the project go awry,- Acceptance of risks that may be broader than the risks normally associated with a particular participant,- The absence of price competition in tender evaluations (for pure alliances),- The need for greater involvement of management resources in the alliance,- No cap on the project schedule or cost,- An increased risk of opportunistic behaviour from other parties to the alliance,- Prohibition on unilateral decision-making, and- Relatively high tendering costs.

Spiral Development/Evolutionary Acquisition(DPPM refers to incremental and spiral)Deviate from the stovepipe/waterfall development lifecycle by incrementally delivering capability. Avoids the big bang approach.Used in IT/systems engineering acquisition where end-user requirements are difficult to define.

Spiral DevelopmentBenefits• Provides the principal with flexibility to change project

requirements at each prototype stage• Incorporates gateway reviews to allow termination or progression

of the acquisition• Provides suppliers with a means to increase or decrease project

costs/fees as risks eventuate or get retired

Costs• Principal is not provided with certainty (fixed cost or fixed

schedule)• Needs greater management resources to implement from both

principal and supplier• There is inherently rework built into this procurement method

Other Options

• Rapid Prototypes – FIRE• Design Competitions/Incentive Prizes• Challenge Based Acquisition

US Government Innovative Contracting Case Studies (2014)

Service Contracts and Performance Based ContractsUsed for:• Maintenance• Cleaning• Catering• Engineering Services

Service Contracts Example Clause (cl 27 of AS4920-2003)The Contractor shall give the Principal reasonable advance notice of when the Contractor needs information, materials, documents or instructions from the Principal.

The Principal shall not be obliged to give any information, materials, documents or instructions earlier than the Principal should reasonably have anticipated at the date of acceptance of tender.

The Principal may direct in what order and at what dates and times the various stages or portions of the Services shall be performed. If the Contractor can reasonably comply with the direction, the Contractor shall do so. If the Contractor cannot reasonably comply, the Contractor shall give the Principal written notice of the reasons.

A performance program is a written statement showing the times and frequency of each performance duration and performance period cycle during the total performance period. It shall be deemed a Contract document.

The Principal may direct the Contractor to give the Principal a performance program within the time and in the form directed.

Does this cater for emergence and complexity?

Leasing Options

Ownership may not be desirable for the following reasons• High upfront costs• Taxation• Loss of flexibility• Obsolescence• Introduction of non-core activities

Options – traditional lease, wet lease, dry lease, hybrids (damp lease)

Leasing Risks and Opportunities

Ownership may not be desirable for the following reasons• Loss of flexibility• ‘Quiet possession’• Potential higher through life costs• Renewal issues

Private sector option – Sale In Lease Out (SILO) with accelerated depreciation.

Public Private Partnerships

‘PPPs may be loosely defined as agreements between the public and the private sector for the purposes of ‘designing, planning, financing, constructing, and/or operating projects, normally regarded as falling within the remit of the public sector.’

Richard Webb and Bernard Pulle, ‘Public Private Partnerships: An Introduction’, Economics, Commerce and Industrial Relations Group, Research Paper 1, 2002/2003.

Public Private Partnerships

Public Private Partnerships - Benefits

• ‘Free’ infrastructure for the public• More efficient delivery• Better utilisation with commercial focus• Eliminate non-core functions

Public Private Partnerships - Costs

• High costs of borrowing• Non-compete clauses• Optimism bias (strategic misrepresentation)• User pays?

Public Private Partnerships - Evolution

Supported Debt Model – the state provides some fundingCredit Finance Guarantee – The state underwrites some of the risk

One Size does not fit all

There is great danger in looking at just one procurement option alone. We need to understand likely contract interfaces, risks, and responsibilities.

- multiparty contracts- multiple contracts

Exercise

Explore the likely contracts required to deliver the Future Submarine Capability- Consider all FIC and inter-agency agreements- Who is the prime?

Implementation issues – Best Practice

• Goal Alignment• Positive Relationships• Prudent Risk Management and allocation• A well crafted acquisition and sustainment strategy• Leadership

must retain robust PM, systems engineering, ILS, and commercial discipline.

Best Practice Goal Alignment

• Gainshare/painshare remuneration arrangements,• Joint decision making,• A collaborative process for managing change,• Integrated product teams, • Partnering charters, and• No disputes/no blame frameworks.

Best Practice Relationship Management

• Best for project culture,• No blame culture,• No disputes,• Transparency and a culture of ‘no surprises’,• Integrated Project Teams, and• Joint decision making.

Best Practice Risk Management

• Early Industry Engagement,• Joint risk and opportunity identification workshops,• Participation by insurance brokers in risk workshops,• Multi stage projects whereby risk allocation varies as

a function of the procurement lifecycle, and• Sharing risks where appropriate.

Best Practice Project Execution Strategy Implementation

• Multi-disciplinary approach• Converge to a solution as more information becomes

available• Strategies are fixed but plans are fluid• Consistency with enterprise objectives

Best Practice Leadership

• Key management participation in the program (participation in monthly reviews)

• Rights of refusal for key positions• A common understanding of the capability

principles is established throughout the organisation (shared vision)

• Talk the talk and walk the walk

Cautionary issues

• Behavioural drivers (too much at risk)• Governance, Accountability, and Probity• Off Ramps (market confidence)

Conclusions and Take Aways

• One size does not fit all in Complex Projects• Plans can be fluid – Strategy should not!• Continually review the Business Case and ensure transparency• Adopt a holistic risk management approach• Craft the Project Execution Strategy to drive the right

behaviours• Foster collaboration• Embrace the chaos! Manage expectations.

Workshops

This afternoon we will conduct some workshops to explore best practice acquisition and sustainment strategies in detail. We will look at three key areas:• Governance• Risk management• Flexibility

Governance

• Tricker’s Model (Compliance and performance)

• Public versus Private Sector Frameworks• Self regulation• Joint Governance

Compliance versus performance

• Compliance – following the rules• Performance – maximising value

– How do we measure performance

Public versus Private sector views

• Compare the Public Governance Performance Accountability Act to the Corporations Act– How does the law treat each entity?

Independent Assurance Review Perspectives

Andrew Pyke

Bringing it TogetherProject Governance in Practice

• A conspiracy of optimism1 (Sealed in a contract)• Complex Project with a huge budget and long schedule• Capability, not just Materiel• Public & Private Sector• Political interest• Public interest

Congratulations………you’re going to be famous! 😱

1 – Source: Royal United Services Institute, 2007

Bringing it TogetherProject Governance in Practice

Flyvberg (1)Project Governance in Practice

• Iron Law of Megaprojects: “Over budget, over time, under benefits, over and over again.”

• Complexity and long time horizons = inherent risk• Often led by inexperienced planners who keep changing over

the course of the project• Decision-making involves many actors and stakeholders with

conflicting interests• Technologies and designs are often non-standard, which not

only makes projects more difficult but persuades managers that their projects are unique and so they don’t learn from others’ experiences

Source: The Oxford Handbook of Megaproject Management, 2017, edited by Bent Flyvbjerg

Flyvberg (2)Project Governance in Practice

• People often commit to projects at an early stage leaving alternative analyses weak or absent

• Large sums of money lead to rent-seeking behavior and optimism bias

• Project scope is likely to change significantly over time• Projects are particularly vulnerable to black swans• Planners rarely account for complexity and black swans, which

is why projects go over budget and under perform• As a result, misinformation about benefits, costs, and risks is

the norm throughout project development

Source: The Oxford Handbook of Megaproject Management, 2017, edited by Bent Flyvbjerg

Solution: Reviews?Project Governance in Practice

Things Said in ReviewsProject Governance in Practice

• “You will do your systems engineering in the end, so you may as well do it now!” (Anon)

• “The first half of your project is saying what you want; the second half is being told what you can have.” (Anon)

• “If you think project reviews are a pain, try an absence of it…….” (Anon)• “Relax: They will never fire you, because then it becomes their problem!”

(Anon)• “A lie can travel half way around the world while the truth is putting on its

shoes.” (Mark Twain)• “What fool planned to buy a $Billion monolith, that takes 10 years to know

whether we can succeed?” (Anon)

A Good ReviewProject Governance in Practice

• Eg OGC Gateway process• Prime Contractor Gate Reviews (various)• Non-Advocate• Principles:

– Non-advocate review– Independence– “How do we get there from here?” (not an audit)– No hidden agenda– Reinforces Responsibility/Authority/Accountability– Review on event, but also periodically (eg annually)

A Good Review Project Governance in Practice

– Not gratuitous “PM 101”– Challenge the “commitment bias” and “groupthink”– Check defensiveness at the door– “What your country has to do for you, not what you can do for your

country”– Line management follow-through– Frank disclosure– Robust discussion– Thorough preparation and analysis– Domain expertise– Collegiate– Presence of key stakeholders

Not a Good ReviewProject Governance in Practice

Source: www.rugby.com.au

Where to from here?Project Governance in Practice

• Investors/owners will no longer tolerate a promise……• Outrage Management:

– Every critic has a megaphone (Social media)– Post-truth world– Expertise is not trusted– Risk is perceived from where you sit (and that’s not where the PM

sits!)

• Complexity itself must be treated• Fragility can be a consequence of over-integration• A good project can’t fix a bad program• AGILE is increasingly understood and accepted

Where to from here?Project Governance in Practice

• Re-thinking value-for-money: fast failure is better value than slow failure

• Minimum Viable Systems – good commercial idea, but can warfighters fight and win with MVS?

• Many problems of the world can’t be fixed and must be coped with, but when do we strategise for this?

• Respect your forebears – they had their own tribulations!• Most projects are run by mere mortals, so stop assuming an

infinite supply of super-heroes• Core questions:

– “Is this Executable?”– “What is the best (least-worst) course of action?”

DiscussionProject Governance in Practice

Self Regulation

• Should we rely on trust alone?– Reputation– Repeat business– Cash Flow– Insurance claims record – Preserve relationships

Case Study – Djimindi Lightweight Torpedo

Background

• Project Alliance between Defence and Torpedo OEM

• Alliance selected• Weapon to be integrated onto five platforms

(FFH,FFG, SH-70B, SH-2G Super Seasprite , AP-3C Orion)

• Subject to two ANAO audit reports

Alliance Features

• No Blame/No liability• Joint decision making (capability board

separate)• Gainshare/painshare

ANAO findings

• Poor initial costings• [the project] will not deliver the capability originally sought by

the ADF, with uncertainty surrounding what will be delivered; has not achieved schedule, with the successful completion of a range of ongoing activities essential to providing certainty regarding when the capability will be released into Navy service

• Sustainment costs much higher than expected (exercise torpedo firing)

What went wrong?

• Explore– Source selection and tender evaluation process– Alliance governance– Conflicts of interest/duty– Motivations– Off-ramps

Risk Management

• Risk attitudes• Negotiated Risk Transfer• Risk and Remuneration

Risk Attitudes

• We need to consider our risk attitudes at the ‘enterprise level’

• What is your attitude to risk?– Risk Neutral (expected value)– Risk Averse – Pessimist (maximin)– Risk Taker - Optimist (maximax)

Customer and Supplier Perspectives

• The development of the acquisition and sustainment strategy should contemplate both the risk attitude of the customer and supplier– What if there is a mismatch?

Case Study

• What delivers ‘best value’– Firm fixed price contract,– Cost reimbursement (with or without a not to

exceed), or– Cost sharing approach

It Depends!

• Cost assurance/cost certainty versus cost control and cost minimisation– Are these mutually exclusive?

We can also apply the same logic to schedule and performance

Cultural issues

• A large organisation should normally adopt expected value decision making but how does this affect individual project managers?

• What can we do to foster a consistent, enterprise wide risk management framework?

Audit Perspectives

• The ANAO is very good at exploring compliance aspects but not performance

“Auditors are the troops who watch a battle from the safety of a hillside and, when the battle is over, come down to count the dead and bayonet the wounded.” - anon

Risk and Remuneration

• Revision– Firm fixed price– Cost reimbursement– Target Cost/Incentive– Cost plus fixed fee

• How does this affect risk?

Case Study – Performance Based Contract

• Fleet of Special Purpose Aircraft• Damp lease• Pay by the hour• Contracted minimum usage level• Surge capacity

Risks

• KPI gaming• Relationships• Safety• Other non-price elements

Solutions to Complex Risks

• Early industry involvement• 2-stage approaches• Joint risk identification• Offer Definition and Improvement Activities

– What are the risks of risk mitigation?

Strategic Supplier Selection

“It's unwise to pay too much, but it's worse to pay too little. When you pay too much, you lose a little money - that's all. When you pay too little, you sometimes lose everything, because the thing you bought was incapable of doing the thing it was bought to do. “ John Ruskin

“A man who knows the price of everything, and the value of nothing.” Oscar Wilde

Key stages

• Approach to market (intelligence)• Selection criteria (link to enterprise objectives)• When does competition end?

Issues

• Asymmetry of information• Bid the A-team, substitute the B-team• Misrepresentation• Mistake• Decoupling Acquisition and Sustainment

Traditional approaches

• Clearly defined specification and statement of work

• Selection based on capability and price

Problems – Tendered prices far exceed initial cost estimates. What of we do not know what we want. What if we over estimate industry capabilities. Loss of innovation

Gate reviews

• Must continually review business case“The business case is continually maintained throughout the life of a project, being formally verified by the board at each key decision point…” Managing Successful Projects with PRINCE 2 (2009) P 22.

Cost as an Independent Variable

• Maximise trade space• Case Study – AIM 9X

– Cost was on equal footing with performance– Established a cost performance IPT– Cost and Operational Effectiveness Analysis

Revealed that little performance gains would be achieved with a new rocket motor and warhead

The problems with templates

• One size does not fit all• Battle between responsiveness and

consistency (smart buyer)• Laziness (emphasis on compliance)• Select the wrong supplier• Sabotage negotiations?

Relational Contracting

• Universal acceptance that collaboration leads to better outcomes– Partnering charters– Gainshare– No disputes– Transparency– Joint risk management

Disputes and Issues resolution

• Resolve at the lowest level possible• Rapid escalation• Alternate resolution (mediation, expert

determination)• Healthy relationship between Sponsors/SROs

Cobb’s Paradox

We know why projects fail, we know how to prevent their failure – why the do they continue to fail?”

What can we do to break the cycle?

• Lessons learned• Culture• Training, education and experience• Expectation management

Conclusions

• Linear problem solving does not always solve non-linear problems

• Focus on enterprise objectives• Do not paint yourself into a corner• Change is inevitable – embrace it!