Upload
cory-terry
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
© Grant Thornton LLP. All rights reserved.
Independent Verification and Validation
Ensuring Project Success
NASACT
August 24, 2015
© Grant Thornton LLP. All rights reserved.
Speakers
Cynthia Dannels, Deputy State Treasurer, SC State Treasurers Office
Tony Hernandez, Principal, Grant Thornton, LLP
2
© Grant Thornton LLP. All rights reserved.
Agenda
• What is Independent Verification and Validation?
• The business case for independent oversight
• Scope of independent oversight
• Elements of effective oversight
• Case Study - South Carolina State Treasurers Office
• Question and answer
3
© Grant Thornton LLP. All rights reserved.
What is Independent Verification & Validation?
Definition
Independent Verification and Validation (IV&V) is a process of employing rigorous methodologies for evaluating the correctness and quality of a project throughout the project lifecycle. Done correctly, IV&V is conducted by a third party organization not involved in the development of the product.
4
© Grant Thornton LLP. All rights reserved.
What is Independent Verification & Validation?
Key Considerations
• Verification – "Did I build the thing right?"
• Validation – "Did what we build work as expected?
• Accreditation – "Should it be used?"
• Realization – "Are we getting the benefits we expected?"
• There is also an underlying implicit principle, and its key
question: Credibility – Should it be trusted?
5
© Grant Thornton LLP. All rights reserved.
The Business Case for Independent Oversight
Business Drivers
• Public sector technology projects are getting larger and more complex, project successes and failures are increasingly more public, and overall projects costs are increasing. This is especially true of large, complex IT acquisitions.
• While the adoption of formal project management methodologies and techniques is becoming more common, many public sector technology projects continue to fail or deliver solutions that are late and over budget.
6
© Grant Thornton LLP. All rights reserved.
The Business Case for Independent Oversight
Critical Success (and Failure) Factors
The reasons why technology projects fail are many and varied, but some root causes appear consistently in troubled projects:
– Lack of a well-defined business case;
– Poorly defined or constantly changing requirements;
– A hurried or badly designed procurement that leads to the wrong vendor being chosen or to a badly written contract;
– The inability to make timely, fact-based decisions; and
– Inadequate skills and experience of government and vendor staff.
7
© Grant Thornton LLP. All rights reserved.
The Business Case for Independent Oversight
Critical Success (and Failure) Factors (cont'd)
It is well accepted in the software development industry that the cost of fixing a defect increases exponentially as a project moves through the development lifecycle.
However, this principle applies equally well to stages of the project that occur long before software development starts. In particular:
• Errors or omissions in the project business case can lead to the wrong solution being procured, so all the project budget may be wasted;
• Errors in the procurement can cause a project to be effectively doomed from the moment the contract is signed;
• Weaknesses in project governance and decision-making can seal a project's fate regardless of the quality of project team efforts.
8
© Grant Thornton LLP. All rights reserved.
The Business Case for Independent Oversight
How Independent Oversight Can Help
When the above types of issues impact projects, it is often because there was a particular element missing from the project governance structure. This element is a party with:
– The experience and foresight to anticipate the problems that could occur in the future;
– The ability and authority to communicate these problems and their consequences to executive management;
– The expertise to recommend a practical course of action to avoid the future problems; and
– The trust of executive management in the objectivity and independence of the recommendations.
An independent oversight consultant can fulfill this role, and can provide project leadership with practical, unbiased advice that can help projects avoid major problems and improve the likelihood of project success.
9
© Grant Thornton LLP. All rights reserved.
The Business Case for Independent Oversight
How Oversight Should Pay for Itself
Independent oversight is analogous to preventative care in the medical industry – a relatively small investment up front can reduce the likelihood of significant expenses later on.
The level of investment in oversight should be a function of the size, complexity and criticality to the mission of the project. An industry average for oversight/Independent Validation & Verification (IV&V) services ranges from 5 - 10% of total project development and implementation cost. Similarly, the cost of IV&V is a function of the size, risk and complexity of a project.
10
© Grant Thornton LLP. All rights reserved.
The Scope of Independent Oversight
Full Project Lifecycle
Issues and risks that impact project success can occur very early in the project lifecycle, and go well beyond the traditional software development activities that are the focus of standards such as the IEEE 1012 standard for IV&V.
For this reason, the scope of independent oversight services should include the full project lifecycle, from initial project concept through sustainment in production operations.
11
© Grant Thornton LLP. All rights reserved.
The Scope of Independent Oversight
Oversight Categories
• Governance – To assess the level of executive stakeholder awareness, buy-in and support, and the willingness and ability to make timely, fact-based decisions.
• Project Management – To determine if the project is planned and managed in a fashion that effectively defines and controls scope, schedule, budget and resources.
• Risk and Issue Management – To determine whether risks and issues are accurately and comprehensively identified and managed. Also to determine whether the culture of the organization supports proactive and transparent communication of risks and issues.
• Quality Management – To assess the activities involved in quality planning, implementation of the quality plan, and the quality assurance functions within the project.
12
© Grant Thornton LLP. All rights reserved.
The Scope of Independent Oversight
Oversight Categories (cont'd)
• Organizational Change and Training – To determine whether internal and external stakeholders are being prepared for – and will be ready for – the impending changes.
• Requirements Management – To assess the requirements management plan and validate that requirements management activities are implemented in accordance with the requirements management plan.
• Procurement and Contract Management – To determine whether procurement activities enable the best possible vendor selection and whether resulting contracts are managed in a fashion that reinforces quality and accountability.
• Solution Development and Implementation –To validate that solutions are being designed, developed and implemented in accordance with approved plans and designs.
• Maintenance and Operations – To determine whether the project is conducting maintenance and operations activities according to plan and service agreements.
13
© Grant Thornton LLP. All rights reserved.
Elements of Effective Oversight
Success Factors and Approach
Oversight success factor High value oversight approach
The experience and foresight to anticipate the problems that could occur in the future.
• Be predictive, not reactive.• Leverage experience to foresee the future consequences
of current decisions and actions.• Findings must be factual and evidence-based.
The ability and authority to communicate these problems and their consequences to executive management.
• Direct line of communication to executive management.• Translate problems and recommendations into language
that is meaningful and relevant to decision-makers.
The expertise to recommend a practical course of action to avoid the future problems.
• Identify pragmatic, actionable solutions to problems.• Be part of the solution, not just an identifier of problems.
The trust of executive management in the objectivity and independence of the recommendations.
• Independence in fact and in appearance.• Integrity and courage of conviction to state the facts, even
(especially) if they are unpopular or uncomfortable.
14
© Grant Thornton LLP. All rights reserved.
Elements of Effective Oversight
Communications Planning
• Communication tools
– Initial review reports
– Status reports
– Briefings
– Dashboards
– Assessments
– Final review reports
– Close-out reports
15
© Grant Thornton LLP. All rights reserved.
Elements of Effective Oversight
Risk Identification and Management
• Risk quantification and reporting
– Risk identification
– Risk analysis and assessment
– Risk rating and ranking
– Risk monitoring and control
– Risk mitigation, escalation and resolution
– Risk reporting
… Enables pro-active decision-making!
16
© Grant Thornton LLP. All rights reserved.
Elements of effective oversight
Key considerations
• Treat the root cause, not just the symptoms.
• Identify pragmatic, actionable solutions to problems.
• Be part of the solution, not just an identifier of problems.
17
© Grant Thornton LLP. All rights reserved.
Case Study
South Carolina State Treasurers Office
Communication toolsOverview of the State Treasurers Office NVEST and DEBT Projects
– Retire the STO's legacy Investment Management System and Debt Management System
– Implement SAP Treasury Management modules for investments (NVEST) and debt management (DEBT)
– NVEST and DEBT Project Go Lives Scheduled for September 2015
18
© Grant Thornton LLP. All rights reserved.
Case Study
South Carolina State Treasurers Office
Communication toolsWhy did you decide to engage a vendor for Independent Verification & Validation services for your project?
19
© Grant Thornton LLP. All rights reserved.
Case Study
South Carolina State Treasurers Office
Communication toolsWhat are some of the barriers to bringing in a vendor for Independent Verification & Validation services?
20
© Grant Thornton LLP. All rights reserved.
Case Study
South Carolina State Treasurers Office
Communication toolsWhat are your expectations of the Independent Verification & Validation vendor?
21
© Grant Thornton LLP. All rights reserved.
Case Study
South Carolina State Treasurers Office
Communication toolsWhat are some of the important considerations for selecting and managing an Independent Verification & Validation services vendor?
22
© Grant Thornton LLP. All rights reserved.
Case Study
South Carolina State Treasurers Office
Communication toolsCan you tell us about the benefits you've seen from the use of Independent Verification & Validation services? Challenges?
23
© Grant Thornton LLP. All rights reserved.
Independent Verification & Validation
Ensuring Project Success
Communication toolsWhat recommendations would you have to share with us?
24
© Grant Thornton LLP. All rights reserved.
Independent Verification & Validation
Ensuring Project Success
Communication toolsWhat trends have you seen related to Independent Verification & Validation services?
25
© Grant Thornton LLP. All rights reserved.
Questions?
26
© Grant Thornton LLP. All rights reserved.
Contact Information
Cynthia Dannels, Deputy State TreasurerT 803.734.2655
Anthony Hernandez, PrincipalT 850.201.7288E [email protected]
Julie Batchelor, DirectorT 919.881.5930E [email protected]
27