by ACIC under FOI · As the P3M3 requires that the lowest of the scores sets the maturity level for a sub model, the maturity assessment for Programme Management is reduced to one

Releas

ed by

ACIC

unde

r FOI

Oakton Consulting Technology CrimTrac

August 2013

Page i

1. EXECUTIVE SUMMARY .......................................................................................................... 2

1.1 Background ............................................................................................................................. 2

1.2 CrimTrac Self-Assessment ....................................................................................................... 2

1.3 Oakton’s Validation Results Summary .................................................................................... 4

1.4 Target Maturity Level Recommendations .......................................................................... . 4

1.5 Independent Assessment ............................................................................................... .... 4

2 THE SELF-ASSESSMENT VALIDATION PROCESS ..................................................................... 5

2.1 The Objective ............................................................................................................ ........... 5

2.2 The Plan .................................................................................................... ..... .. ... ............ 5

3 PROCESS REVIEW ..................................................................................... ...... .................... 7

4 SELF-ASSESSMENT VALIDATION ........................................................ .... ........................... 10

4.1 Portfolio Management Level 2 ................................................... .... ................................ 10

4.2 Programme Management Level 1............................................ .... ..................................... 12

4.3 Project Management Level 2 ............................... .............................................................. 14

4.4 P3M3™ Sub-Model Process Perspective Score .... ...... .................................................. 17

4.5 P3M3™ Generic Attributes Scores ......... ............... ........................................................... 17

5 AGENCY TARGET MATURITY LEVELS ............. ........ ........................................................... 19

5.1 Maturity Targets ............................ ..... ......................................................................... 19

5.2 Gap Analysis ...................................... .................................................................................. 19

6 RECOMMENDATIONS ............ ..... .................................................................................... 21

6.1 Benefit’s Managem nt ...... ....... ........................................................................................ 21

6.2 Stakeholder Engagemen ..................................................................................................... 21

6.3 Risk Management . .............................................................................................................. 21

6.4 Resource Management ......................................................................................................... 21

6.5 Organisational Governance .................................................................................................. 22

6.6 Programme Management ..................................................................................................... 22

ATTACHMENTS .......................................................................................................................... 23

ATTACHMENT A - P3M3™ MODEL AND DEFINITIONS .................................................................. 24

ATTACHMENT B - TERMINOLOGY ............................................................................................... 27

Releas

ed by

ACIC

unde

r FOI

Oakton Consulting Technology

Government

Consulting Technology

Oakton Services Pty Ltd ABN 31 100 103 268

Canberra Unit 2 45 Wentworth Avenue Kingston ACT 2604 Australia t +61 2 6230 1997 f +61 2 6230 1919

www.oakton.com.au

1. Executive Summary

1.1 Background

CrimTrac was established through an Inter-Governmental Agreement in 2000 as a collaborative partnership between the Commonwealth, States and Territories to help combat crime in Australia. CrimTrac’s aim is to generate information sharing solutions across police services, the wider law enforce ent ommunity and international counterparts, whilst its current business priorities include Law Enfor ment Information, Background Checking, Biometrics and Capability Sustainability and Governance

Since its establishment, CrimTrac has adopted a structured, tailored approach to Portfolio, Programme and Project (P3) Management, based on the Office of Government Commerce (OGC) methodologies for project (PRINCE2) and programme management (Managing Successful P ogrammes). CrimTrac has established a Project Management Office (PMO) which is developing a framework for d livery of P3 Management.

CrimTrac is a unique agency, somewhat different to a standard FMA Act portfolio agency, as it is essentially the information technology service developer and provide for shared information services for its business areas which reside within the Federal, State and Territory Pol e agencies. This creates some unique challenges for CrimTrac’s execution of Portfolio, Programme and Project Management.

1.2 CrimTrac Self-Assessment

In June 2013 the CrimTrac Project Man gement Office (PMO) conducted a survey to self-assess the maturity level of CrimTrac’s P3 Managemen The elf-assessment was based on the OGC’s self-assessment questionnaires for Portfolio Programme and Project Management. This self-assessment was a follow up to a previous self-assessment undertaken in 2012.

The 2013 survey was comp eted by a cross section of 22 of CrimTrac’s Directors, Project and Programme Managers. Oakton’s review of this assessment indicates that it was well managed and appears to reasonably reflect the existing level of maturity for most of the process perspectives. We have identified an issue with the overall assessment for Programme Management Maturity which is discussed below.

Portfolio Management Maturity - Repeatable

Perspective CrimTrac Self-assessment

2012

CrimTrac Self-assessment

2013

Oakton Validation 2013

Management Control 3 3 3

Benefits Management 2 2 2

1 CrimTrac Strategic Plan 2015

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Portfolio Management Maturity - Repeatable


2012


2013


Financial Management 3 3 3

Stakeholder Engagement 3 3 3

Risk Management 2 2 2

Organisational Governance 3 3 3

Resource Management 2 2 2

Programme Management Maturity - Aware


2012


2013




Financial Management 3 3 1





Project M nagement Maturity - Repeatable


2012


2013




Financial M nagement 3 3 3





Note that the overall assessed maturity level of Portfolio, Programme and Project Management is equal to the lowest score for the process perspectives.

The CrimTrac self-assessment rated Portfolio, Programme and Project Management assessed at Maturity level 2 – repeatable processes.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

The definition of maturity level 2 is set out below:

Maturity Level 2 – Repeatable Process

The Agency ensures that each programme and/or project in its portfolio is run with its own processes and procedures to a minimum specified standard.

Application of a consistent method across all change initiatives is not yet observed.

1.3 Oakton’s Validation Results Summary

As indicated in the tables above, Oakton’s validation process supported the bulk of the assessments except for reductions in the assessments for:

Management Control, Benefits Management and Financial Management at Prog amme level from levels two, two and three respectively to level one, and

Resource Management at Project level from three to two.

As the P3M3 requires that the lowest of the scores sets the maturity level for a sub model, the maturity assessment for Programme Management is reduced to one. The definition of maturity level one is set out below.

Maturity Level 1 – Awareness of Process

The Agency recognises programmes and runs them differently from projects.

1.4 Target Maturity Level Recommendations

Oakton recommends that CrimTrac set targets for delivery maturity levels as follows:

Portfolio = Level 3: Defined Processes

Programme = Level 2: Repeatable Processes

Project = Le el 3: Defined Processes

Capability delivery improvements across CrimTrac’s Portfolio, Programme and Project Management processes can lead to a more centralised and defined framework with mature investment management processes. Section 6 outlines Oakton’ recommendations for improvement.

1.5 Independent Assessment

This Indepe dent Assessment was conducted by APMG Registered Consultant (PPMRCAU/026), and , Senior Consultant.

s 47Fs 47

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

2 THE SELF-ASSESSMENT VALIDATION PROCESS

2.1 The Objective

CrimTrac engaged Oakton to validate the internal self-assessment conducted in June 2013. CrimTrac asked Oakton to deliver a report by 31 July 2013 that is an independent validation of the self-assessment. The report is to provide a thorough explanation of rationale and outcomes of the independent assessment, highlighting any weaknesses or recommended improvements which CrimTrac could make in the future.

2.2 The Plan

Oakton conducted this review using the APMG P3M3™ Assessment Tool in accordance with APMG guidelines and practices. The following plan outlines the stages, key activities and deliverables of the assignment.

Review and validation was carried out via these four stages:

S age Key Activities

Initial Meeting Determine scope and expectations

Agree validation scope and coverage

CrimTrac to arrange interview schedule for a selected sample of interviewees from original self-assessment

Process Review Review self-assessment processes, dependant of which P3M3 self-assessment model was used

Review prior P3M3 assessments / benchmarks

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Review existing documentation, processes relevant to perspectives covered by the self-assessment

Data and Analysis Validation

Validate self-assessment conclusions against findings from the review stage, prior assessments and self-assessment data

Carry out spot check interviews2 with a subset of the interviewees or survey participants

Review conclusions from internal self-assessment

Report and Findings Identify and document gaps between self-assessment conclusions, data and Validation findings.

Endorse validation report

Finalise P3M3 Self-assessment Independent Validation Assessment Report

Present report and findings if required.

Deliverable: Assessment Report

2 Interviews were conducted on the basis that the individual interviewee(s) would not be identified.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

3 PROCESS REVIEW

The CrimTrac P3M3 Self-assessment process was based on the guidance provided by the UK Office of Government Commerce handbooks for P3M3 self-assessment. OGC has produced a handbook for each sub-model of Portfolio, Programme and Project Management. The figure below shows the P3M3 sub-models and process perspectives.

For each process perspective for each sub model, the self-assessment questionnaire asked assessors to select the most appropriate description of maturity level for that process perspective.

The PMO provided the que tionnaire o a representative sample of CrimTrac Directors, Programme and Project Managers for respons The results were assessed from 22 respondents.

Oakton has reviewed these responses and apart from noting that there were a few individual assessments of maturity at the higher levels of Level 4 (managed processes) and Level 5 (optimised processes) that appear hard to justify, th assessments appear to reflect a good understanding of the current state of affairs at CrimTrac. It is worth noting that removing these outlying scores does not significantly change the overall assessment of CrimTrac’s maturity.

Oakton not s that in reporting results of the self-assessment the PMO has provided fractional scores – e.g. a maturity level of 2.4 for Stakeholder Engagement for Project Management. Whilst these scores provide some indication of movement between the 2012 and 2013 self-assessments, this approach is not supported by the Methodology. The approved approach to scoring is to determine the median score for a process perspective and round the median score to the nearest whole number. The overall maturity assessment for each sub-model is then determined from the lowest process perspective score.

Based on the 22 responses, the CrimTrac P3M3 Self-assessment rates CrimTrac’s maturity level at Level 2 – repeatable processes for each sub-model. Individual Process Perspective scores are shown in the tables below, taken from the CrimTrac Self-assessment Report Rele

ased

by A

CIC un

der F

OI

Releas

ed by

ACIC

unde

r FOI

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

4 SELF-ASSESSMENT VALIDATION

In addition to reviewing the self-assessment process, Oakton also reviewed existing CrimTrac documentation and processes and interviewed a sample of eight people who could represent views from the PMO, Portfolio, Programme and Project Management.

The Oakton approach was to conduct the interviews and document review as we would do for a standard P3M3 assessment. We did not discuss individuals’ responses to the self-assessment questionnaire with them. We also matched documentary evidence with the expected level of evidence outlined in the APMG guidelines for P3M3 assessments.

Following are the assessment results with a description of the rationale behind the assessmen .

4.1 Portfolio Management Level 2

Overall Portfolio Management maturity is assessed as Level 2.

Portfolio Management is at level 2 and not level 1 because:

Across process perspectives there is a recognised and re eatable set of processes.

Some processes display a considerable level of matu ity, in particular there is a strong alignment of change initiatives to strategic objectives and priorities. Financial management show signs of maturity and internal management controls display the right approach to coordination of portfolio level activities across the organisation.

Portfolio Management is at level 2 and not level 3 be ause:

Benefits realisation activities are managed to level 2. Although benefits are identified in business cases there is little evidence of benefits ma agement including benefits ownership and tracking being consistently applied at the portfolio level.

Risk Management is ass ssed a level 2 as risk management across the portfolio is not based on a common, centrally managed pro ess. Portfolio risks appear to consist of rolled up project risks and are not yet linked t enterpris operational risks.

Resource Management s assessed at level 2. Although there is a resource allocation process managed within the PMO, this is currently restricted to a couple of resource categories and does not address the full re ource implications of portfolio decisions.

4.1.1 Management Control Level 3

Management Control within Portfolio Management was found to be at level 3 and not level 2 because the assessment revealed that the portfolio management processes are centrally defined and understood, as are the roles and responsibility for delivery. The roles of the executive committees including the Portfolio Board are well understood, as is the process for approval of new initiatives and ongoing monitoring of programmes and projects. The Strategy Team and the Project Management Office support the portfolio management process with idea assessment, reporting and independent assurance. There is a clear link between the value assessment contained in the business cases for each new initiative which drives the annual plan and associated priorities that are used by the executives to develop a portfolio of projects and programmes. All new initiatives are centrally considered and approved before becoming a part of the change portfolio.

Management Control within Portfolio Management was found to be at level 3 and not level 4 because the assessment found no evidence of metrics used to measure the performance and success of the portfolio.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

4.1.2 Benefits Management Level 2

Benefits Management within Portfolio Management was found to be at level 2 and not at level 1 as there is a reasonable level of understanding of the need for benefits management. Benefits are identified at project level and are contained in the business case documents.

Benefits Management within Portfolio Management was found to be level 2 and not level 3 because there is little evidence of a defined benefits realisation and management process being exercised by the Portfolio Board. Recognition of benefits happens consistently in the business case documents, however, the quality of those benefits needs improvements with focus on the measurable aspect of those benefits and assignment o ownership and accountability for realisation. It is recognised that the relationship of CrimTrac to its jurisdictional policing stakeholders makes benefit ownership and measurement a challenge for CrimTrac projects and programmes.

4.1.3 Financial Management Level 3

Financial Management at the Portfolio level was found to be at level 3 and not level 2 be ause there are established standards for the investment management process and the preparation of business cases. In addition, portfolio investment costs are monitored by the Portfolio Board. Annual financial planning exists. Regular reviews of the portfolio occur and corrective actions are put in place to rectify under or over spend situations. There was some evidence of projects being stopped, due to reassessment of the business case.

Financial Management within Portfolio Management was found o be level 3 and not level 4 because there is little evidence of a proactive evidence based management of the osts. Cr mTrac does not exercise the prediction of future financial performance based on the metrics gath ed for programmes and projects, e.g. “estimates to complete” at the programme and project level are n t assessed.

4.1.4 Stakeholder Management Level 3

Stakeholder Management at the Portfolio level wa found to be at level 3 and not level 2 because external stakeholders are engaged through the Strategic Issu s Group and Board of Management, project/programme based user groups and domain specific forums e.g. CFO forum. CrimTrac has a standard documented approach to identifying, managing and comm nicating with portfolio stakeholders. Project and Programme Managers are required to comply and be consistent with a Portfolio level Communications Strategy and Agency Communications Principles and Guidelines.

Stakeholder Management within Portfolio Management was found to be level 3 and not level 4 because there is no evidence of quantitat ve analysis of the effectiveness of communications and stakeholder engagement.

4.1.5 Risk Management Level 2

Risk Management at the Portfolio level was found to be at level 2 and not level 1 as there is a risk management framewo k for projects that result in project risks being rolled up to portfolio level. However there is a view that th implications at Portfolio level are not well understood and currently there is poor integration with ent rprise/operational risks.

Risk Management within Portfolio Management was found to be level 2 and not level 3 because there is not a standard documented approach to identifying, quantifying and managing portfolio risk. Currently, Portfolio risk is simply the reporting of higher level project risks to the Portfolio Board. There is no identified role to manage and monitor portfolio risks.

4.1.6 Organisational Governance Level 3

Organisational Governance at the Portfolio level was found to be at level 3 and not level 2 because there is a clear alignment of initiatives, programmes and projects with the strategic objectives. Projects have been

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

stopped as a result of the review at the Portfolio level. All interviewees have a clear understanding of portfolio governance and the process for approval of an initiative.

Organisational Governance within Portfolio Management was found to be level 3 and not level 4 because portfolio management has only recently been established and there is no evidence of collection of portfolio performance information to support Portfolio Board decisions and risk mitigation.

4.1.7 Resource Management Level 2

Resource Management at the Portfolio level was found to be at level 2 and not level 1 because the resour e management process is being developed and extended to cover more resources. The resource needs of individual projects are provided in business cases. There is a resource allocation process for Project Managers and Business Analysts (which is being extended to other disciplines).

Resource Management within Portfolio Management was found to be level 2 and not level 3 b ause there is no evidence of resource management being used to develop targeted resource capabilities to enable delivery of longer term priorities. There also appeared to be limited use of commercial resource providers to supplement CrimTrac resources to meet peak demands.

4.2 Programme Management Level 1

Overall Programme Management maturity is assessed as Level 1

Programme Management is at level 1 and not level 0 because:

Key individuals were found to have practical delivery expe ience in programme management. The interviews indicated some understanding of p ogrammes, their specific management practices and their potential value to the organisation

Programme Management is at level 1 and not level 2 because:

There is no evidence of a CrimTrac Programme Management Framework. The current version of the Portfolio, Programme and Project Governance (P3G) Framework includes Program Management Boards, but they appear to act as mo e senior level Project Boards for groups of projects.

Whilst the assessors obser ed st ong control over budget and expenditure at project and portfolio levels, there is little or no finance provision for work at the programme management level. For example, for a strategic change program better practice would provide for programme level funding for the roles and activit es of the Programme Manager, Business Change Manager and perhaps programme support.

There was no vidence of strategic change programmes being established as a result of a programme level busines case.

The e appears to have been a retreat from a previous commitment to adopt Managing Successful Programmes as a “standard” approach to programme management within CrimTrac.

A number of programmes appear to be a grouping of similar projects managed as an ongoing program of work, rather than a programme management approach to strategic change.


Management Control within Programme Management was found to be at level 1 and not level 0 because programme management terminology is used by some members of the agency and some experienced programme managers have been engaged. Programme Management, as currently implemented within the agency consists of coordination of multiple projects. There is regular reporting to the Programme Boards and the Portfolio Board.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Management Control within Programme Management was not at level 2 because there is little evidence of programme management guidance in CrimTrac’s P3G Framework. It is understood that some Programme Management material was removed from the current version of the P3G Framework as a result of the recommendations from the Review of Internal Governance that was performed by Pcubed in 2012. There is concern among some managers within CrimTrac about the appropriateness of applying Programme Management methods, as it is seen as an unnecessary level of governance for such a small agency.

The Framework does not identify programme-specific deliverable templates, for example programme management plan, programme blueprint, project dossier, and programme business case.


Benefits Management within Programmes was found to be at level 1 and not at level 0 because i terviewees recognised the need to identify and track benefits. There was evidence of links between strategy and the initiation of projects. Both outcomes and benefits are described. However, their measurem nt criteria and tracking processes are underdeveloped.

Benefits Management within Programme Management was found to be level 1 and not level 2 as there was no evidence of Programme level business cases. Realisation of benefits throughout the programme lifecycle was not demonstrated and programme benefits were not identified beyond aggregation of project benefits. There was some evidence that project benefits were revisited and verified after proj ct c osure, however the quality and availability of benefit metrics is questionable.


Financial Management at the Programme level was fo nd to be at level 1 because there was no evidence to indicate that finances were managed at programme level. Busi ess cases and budgets were all approved at project level and none of the programmes could p int to any programme budget. For example the Firearms Programme Manager position is funded from project budgets. There was no evidence of Programme level budget for managing change. Although programm finances are well managed and controlled at portfolio and project level, the non-existence of programme level budget makes it difficult to provide a higher level of maturity assessment.


Stakeholder Management within Programmes was found to be at level 2 and not level 1 because there was some evidence of a structured approach to identify, manage and communicate with stakeholders. Stakeholder engagement is a planned activity.

Stakeholder Management within Programme Management was found to be level 2 and not level 3 because stakeholder management is not exercised in accordance with a centrally defined process. There is no structured centrally managed communications plan to balance communications within all programmes and projects. Local knowledge is used in place of specialist communication skills to design and assess communication activities. The costs associated with communications and stakeholder management is not considered explicitly or separated in the programme costs.

4.2.5 Risk Management Level 2

Risk Management at the Programme level was found to be at level 2 and not level 1 because Risk Management is based on centrally defined processes and roles used by all projects. Risks are reviewed and escalated through-out the project lifecycle. Responses are approved using this process.

Risk Management at the Programme level was found to be level 2 and not level 3 because although risks are assessed for individual projects there is not a standard approach to identifying and managing programme risks.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Programmes tend to report project level risks, rather than programme level risks. One programme is beginning to manage Programme level risks.


Organisational Governance at the Programme level was found to be at level 2 and not level 1 because there are decision making controls being applied in a repeatable manner through the Programme Boards. There is evidence of the links between organisational strategy and the direction of the programme.

Organisational Governance within Programme Management was found to be level 2 and not level 3 becaus , while there were controls in place for the CrimTrac’s major programmes, standards for Programme governance structure or decision making were not evident in the CrimTrac P3G Framework. The role of business change managers in programmes was not evident.


Resource Management at the Programme level was found to be at level 2 and not leve 1 because the utilisation of resources, people specifically, is monitored in the PMO in order to plan, manage and meet the resource needs for all approved initiatives.

Resource Management within Programme Management was found to be level 2 and not level 3. While CrimTrac has elements of a resource management framework in place, sharing of resources between projects and programmes is often undertaken by negotiation for individual staff on a case by case basis. There is no evidence of resource management being used to develop targeted resou ce capabilities to enable delivery of longer term priorities. There also appeared to be limited use of commercial resource providers to supplement CrimTrac resources to meet peak demands.

4.3 Project Management Level 2

Overall Project Management maturity is assessed as Level 2.

Project Management is at level 2 and not level 1 because:

CrimTrac recognises projects (their language, processes and framework) and run them differently to business as usual activities based on a standard Project Framework.

There is some consistency in approach across projects with project managers, as all recognise or try to apply the current P3G Framework. There is an embryonic Project Management community of practice for sharing accepted practices across projects.

Key individuals have practical delivery experience in project management. New project managers are supported in their role through the community of practice and the PMO.

Project Management is at level 2 and not level 3 because:

Whilst there is a project management framework developed by the PMO, there are still gaps in the f amework, and project execution in the areas of Benefits Management, Stakeholder Engagement and Resource Management.


Management Control within Project Management was found to be at level 3 and not level 2 as all projects have designated project managers who use the project management framework and information system. Issues and risks were monitored through a standardised process. Change control is applied where appropriate.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Management Control within Projects was not at level 4 because project management metrics were not used to monitor process capability and manage project performance in quantitative terms.


Benefits Management within Projects was found to be at level 2 and not at level 1 because there is recognition of the concept of benefits versus output from projects. Outcomes and benefits are recognised as an element within business cases and project plans.

Benefits Management within Project Management was found to be level 2 and not level 3 because while there is some documentation of benefits, measurement, on-going and post project tracking is not defined and explicit. Business ownership of Benefits Realisation is not normally defined in project documents. Benefits are usually not estimated in financial terms against centrally managed assessment criteria. The Proje t Management Framework needs some development in terms of the Benefits Management by providing practical guidance in that area.


Financial Management at the Project level was found to be at level 3 and not level 2 b cause project business cases documents are always developed and approved at project initiation using a s andard process. A template and guide to developing a business case is centrally provided. Busine cases are reviewed at appropriate points within a project’s lifecycle, and action taken to keep proje ts on track. Approved Project Mandates are required to start a project and these define the scope and budget allocation for the project. Signoff is required for any additional costs imposed by the project, including acces ing c tingency allowances.

Financial Management within Project Management was found to be level 3 and not level 4 as process tolerances were only monitored against total project funding. E rned value concepts for monitoring a project are not in place and risks are not evaluated in financial terms.


Stakeholder Management at the Project leve was found to be at level 2 and not level 1 because a standard approach to identifying, managing and ommunicating with stakeholders exists and is used to identify and plan engagement. This was done in a ba ic and consistent way. Commitment to meeting stakeholder needs and actively involving stakehold rs in proje ts was evident among interviewees.

Stakeholder Management within Project Management was found to be level 2 and not level 3 because although there is evidence f documented Communication Plans for some projects, there does not appear to be a standard appro ch used by all projects. Corporate communications is not involved in development of stakeholder communi ation, coordinating key messages across projects or enhancing the effectiveness of communication activi y.

4.3.5 Ri k Management Level 2

Risk Management at the Project level was found to be at level 2 and not level 1 because Risk Management is applied by all project managers using consistent templates and tools. A risk and issues log is maintained by all major projects and is held within a central information system. Standard risk management tools are applied and risks are identified with stakeholders. Mitigation actions and risks are escalated to higher governing bodies where required. Project managers view risk in terms of the internal impact of the project as well as its organisation-wide impact. The PMO provides support for Risk Management by facilitating risk workshops for new projects.

Risk Management within Project Management was found to be level 2 and not level 3 as although standard templates and tools were used to record and manage risks, there was little evidence of regular risk reviews, or audits of risks within projects.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au


Organisational Governance at the Project level was found to be at level 3 and not level 2 because there is clear evidence of a centrally defined set of agency level controls applied to projects. Regular consolidated reporting occurs with a single source of the truth through the CA Clarity tool. Terms of Reference and clear reporting lines to governance bodies are maintained. Decisions of Board meetings are minuted and auditable. The Project Status Reports report progress against project outcomes. A tiered system, for classification of projects, assists in ensuring that projects are evaluated against consistent criteria prior to project approval.

Organisational Governance within Project Management was found to be level 3 and not level 4 because her are few indicators of understanding the impact of projects on business performance. Decisions to launch projects do not usually consider the impact on current initiatives.


Resource Management at the Project level was found to be at level 2 and not level 1 because here is a recognition that the Agency needs to manage resources effectively to enable successful delive y of projects. Human resource planning is carried out at the individual project level and this is ex ended to planning across groups of projects within the PMO.

Resource Management within Project Management was found to be level 2 and not level 3 because although there is a centrally defined human resource management process operated by the Resource Allocation Committee that coordinates project resource planning and allocation, there was no evidence of resource utilisation and performance tracking being available. There was no evidence of resource management for other than human resources.

The coverage of resources by the RAC is gradually bein extended from Project Managers and BA’s to other specialist disciplines. There was some collaboration at the agen y level to share critical or limited resources. This was driven through negotiation rather than a entrally maintained resource plan across the CrimTrac.

Releas

ed by

ACIC

unde

r FOI

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Generic Attributes Maturity

Generic Attribute

Assessment Maturity Level

Recommended Action

Roles and Responsibilities

Roles and Responsibilities are defined in some areas but not across the agency

2 Develop centrally managed role definitions and competencies – use to recruit.

Experience in P3 Management Key individuals may have

practical delivery experience and track record

2

When recruiting contract and permanent staff for P3M roles ensure that they have relevant practical experience and tr ck record in successful delivery.

Capability Development

Generic training may be provided in key concepts and there may be individuals undertaking qualification training. Local sharing of knowledge may exist but is mostly ad-hoc

2

Consider agency s ecif training in key areas. Formalise he current informal Project Management forum and exten its a tivities to improve knowledge of planning and estimating and benefits management.

Planning and Estimating Processes

Plans exist but are not underpinned by a consistent development methodology. There are no standard techniques for estimation, o capture of estimating history to support improvements.

2

Develop agreed estimating t chniques for CrimTrac projects. Capture estimates and validate on project closure to refine the methods.

Information and Documentation

Agency wide information standards on conf dentiality, availability and integrity. Formal information release procedur s. Centralised storage and ccessible information.

3

Maintain progress in information management using Clarity and proposed EDRMS.

Scrutiny and Review

L cal reviews with some corrective actions undertaken within the group.

2 Apply independent reviews to key projects at critical stages.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

5 AGENCY TARGET MATURITY LEVELS

5.1 Maturity Targets

Oakton recommends that CrimTrac set target capability delivery maturity levels as follows:

Portfolio: Level 3 - Defined Processes

Programme: Level 2 - Repeatable Processes

Project: Level 3 - Defined Processes

5.2 Gap Analysis

The following analysis identifies the process perspectives that are responsible for t e ma urity level score i.e. the low scoring process perspectives. The Capability Improvement plan needs to focus on improving these areas and addressing the ‘gap’.

Portfolio Level 2 => 3: Defined Processes

Management Control Level 3 => 3

Benefits Management Level 2 => 3

Financial Management Level 3 = 3

Stakeholder Management Leve 3 => 3

Risk Management Level 2 => 3

Organisational Governance evel 3 => 3

Resource Managemen Le el 2 => 3

Improvement in Benefits Management Risk Management and Resource Management are required in order to achieve a level 3 maturity lev l

Programme: Level 1 => 2: Repeatable Processes



Financial Management Level 1 => 2

takeholder Management Level 2 => 2


Organisational Governance Level 2 => 2

Resource Management Level 2 => 2

Improvements in Management Control, Benefits Management and Financial Management are required. Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Project: Level 2 => 3: Defined Processes



Financial Management Level 3 => 3

Stakeholder Management Level 2 => 3


Organisational Governance Level 3 => 3

Resource Management Level 2 => 3

Improvements in Benefits Management, Stakeholder Management, Risk Management and Resource Management are required.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

6 RECOMMENDATIONS

In order to improve CrimTrac’s P3M maturity levels Oakton recommends that CrimTrac focus on improvements in the following areas:

6.1 Benefit’s Management

CrimTrac has a good understanding of Benefits Management. It could improve its maturity in this area by developing guidance for P3M staff tailored to CrimTrac’s unique situation with regard to Benefits real sation in the jurisdictional policing agencies. Some basic steps to improve the definition of benefits and develop practical metrics would assist in improving the tracking of benefits.

We also recommend that when establishing project and programmes, the formal appoin ment of appropriate benefit owners should be part of the process. This would also encourage deeper stakeholder communication with projects/programmes and commitment to benefits realisation.

If CrimTrac wishes to improve Programme Management Maturity, then development of Programme Management level business cases which include identification of Programme Be efits would assist.

6.2 Stakeholder Engagement

As indicated earlier, Oakton recognises the unique situ tion of CrimTrac as an IT service provider to jurisdictional policing agencies. That said, it appears that there s an opportunity to improve stakeholder engagement at programme and project level, by improving the identification of and engagement with business owners/users of the outputs of CrimTrac projects It may require increasing budgets for engagement and business change within these agencies (at the cost of doing fewer projects).

This recommendation may require significant strategic level engagement with the Strategic Issues Group and/or Board of Management. The risk of no acting is that CrimTrac will remain an IT provider rather than a business partner delivering signif cant va ue to he jurisdictions.

6.3 Risk Management

The existence of Risk Management frameworks for P3M and the Enterprise Risk Management Framework, based on AS/NZS ISO 13000 standard has established the foundations for a mature Risk Management process within CrimTra Capture of all project risks in Clarity is a positive step. Additional effort is recommended to ensure that programme and portfolio level risks are identified and managed separately from escalated project risks. It may be ppropriate for the Portfolio Board to regularly devote a meeting to consider enterprise and portfolio level isks and mitigation strategies.

Impr vements in Risk Management at project level could involve more formal risk reviews at key project gates.

6.4 Resource Management

CrimTrac has established a Resource Allocation Committee and developed a process for projects to seek required resources. It is recommended that CrimTrac make more immediate use of external resources (contractors) to fill temporary skill gaps in preference to seeking expressions of interest from internal staff, which merely moves the gap from one project to another. Further work should be done to improve planning for future capability.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

At Portfolio level resource capacity and capability should be made more visible when deciding on priorities for project/programme initiatives.

6.5 Organisational Governance

Whilst conducting this assignment Oakton became conscious of the heavy governance load currently being born by the Portfolio Board. The time required and the level of detail provided for recent PFB meetings indicates a need to streamline this process and push decision making down to more appropriate levels. he PFB’s focus should be on setting strategic direction, setting priorities, managing enterprise and portfolio le el risks and monitoring performance and managing exceptions. Its current focus seems to be in the detail of minor changes to projects, which could be better managed at programme or project board level.

6.6 Programme Management

As indicated in our assessment of Programme Management maturity, there is a fundamen al issue with the definition of Programme and how CrimTrac wishes to manage programmes. At the moment two of the three programmes appear to be themed aggregations of projects. The Firearms Programme is attempting to set itself up as a true programme of strategic change, but with competing views within CrimTrac about the applicability of programme management, seems to be struggling. CrimTrac executive should make a conscious decision either to use a programme management approach for it major change programs, or to avoid major business change and just run IT projects. If the latter approach is preferre , then we would recommend that Programme Management be removed from the P3M3 assessm nt p oc ss altogether.

Alternatively, if CrimTrac executive do see a place for Programme Management (as per the OGC, Managing Successful Programmes model) then CrimTrac should d velop a Programme Management Framework, in consultation with the jurisdictional policing agencies who will be the business owners of business change initiatives. As a first step we suggest that the Firearms P ogramme be used as a pathfinder to establish the artefacts and processes to be used for managing C mTrac Programmes. This should include making financial provision for programme management, and busines change management in addition to agreed funding for the existing projects in the Firearms programme.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Attachments

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

ATTACHMENT A - P3M3™ MODEL AND DEFINITIONS

P3M3™ Model

The following terms are used in the assessment of portfolios, programmes and projects.

Portfolio management

A Portfolio is the totality of an organisation’s investment (or segment thereof) in the changes required to achieve its strategic object ves.

Portfolio management is a coordinated collection of strategic processes and decisions that together enable the most effective balan e of o ganisational change and Business as Usual (BAU).

Programme M nagement

A programme is a temporary, flexible organisation created to coordinate, direct and oversee the implementation of a set of related projects and activities in order to deliver outcomes and benefits related to the organisation’s strategic objectives.

Progra me management is the coordinated organisation, direction and implementation of a dossier of rojects and transformation activities to achieve the outcomes and realise the benefits that are of strategic

importance to the business.

Project Management

A project is a temporary organisation, usually existing for a much shorter duration (than a programme), which will deliver one or more outputs in accordance with a specific Business Case. A particular project may or may not be part of a programme.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Project management is the planning, delegating, monitoring and control of all aspects of the project, and the motivation of those involved, to achieve the project’s objectives with the expected performance targets of time, cost, quality, scope, benefits and risks.

The characteristics and elements of each process perspective are outlined below.

Management Control

Management control is characterised by clear evidence of leadership and direction, scope, stages, tranches and review processes during the course of initiatives. There are regular check points and clearly defined decision making processes. There are full and clear objectives and descriptions of what the initiative will deliver with a blueprint of the target operating model.

Benefits Management

Benefits management is the process that ensures the desired business change outcomes are c early defined, are measurable and are ultimately realised through a structured approach and with full organisational ownership. Benefits are assessed and approved by the organisational areas that will deli er th m. Benefit dependencies and other requirements are clearly defined and understanding gained n h w outputs of the initiatives will meet those requirements. There should be evidence of suitable class fication of benefits and holistic view of the implications being considered. All benefits should be owned, have realisation plans and be actively managed to ensure they are achieved. There is a focus on operational tran ition with review and follow up on actions needed to ensure benefits are owned and realised.

Financial Management

Financial management ensures that the likely costs of the initi tive are captured and evaluated with a formal business case and that costs are categorised and managed over t e investment lifecycle. There should be evidence of involvement of the Agency’s financial func ions, with approvals imbedded in the broader organisational hierarchy. The business case should defi e the value of the initiative to the business and contain a financial appraisal of the possible options. The bu iness case will be at the core of decision- making during the initiative’s lifecycle, and be linked to formal review stages and evaluation of the costs and benefits associated with alternative actions. Financial management will schedule the availability of funds to support the investment decisions.

Stakeholder Management

Stakeholders are the key to the suc ess o any nitiative. Stakeholders at different levels within and outside the organisation will need to be analysed and engaged effectively in order to achieve the objectives in terms of support and engagement. Stakeholder engagement includes communications planning, the effective identification and use of d fferent communication channels, and the techniques to enable the objectives to be achieved.

Stakeholder engagem nt is an ongoing process across all initiatives and is inherently linked to the initiative’s lifecycle and gove na ce controls.

Risk Management

Risk management provides a view on the way opportunities and threats are presented by the initiative. Risk management maintains a balance of focus on threats and opportunities with appropriate management actions to minimise or eliminate the likelihood of any identified threat occurring, or minimise the impact if it does occur, and to maximise opportunities.

Organisational Governance

This looks at how delivery of initiatives is aligned to the direction of the organisation. It considers how start-up and closure controls are applied to initiatives and how alignment is maintained during an initiative’s lifecycle. This differs from management control which views how control of initiatives is maintained internally. This perspective looks at how external factors that impact on the initiative are controlled if possible or mitigated if not. The management of these external factors are used to maximise the final result. Effective sponsorship is critical to this.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

Resource Management

Resource management covers all types of resources required for delivery. These include human resources, buildings, equipment, supplies, information, tools and supporting teams.

A key element in the management of resources is the process of acquiring resources and how supply chains are utilised to maximise effective use of resources. There will be evidence of capacity planning and prioritisation to enable effective resource management. This would also include performance management and exploitation of opportunities for greater utilisation. Resources capacity considerations will be extended to the capacity of the operational groups to resource the implications of change.

Maturity Level Definitions

Maturity Level 1 - Awareness of Process

Processes are not usually documented, actual practice is determined by events or individual preferences, and performance is variable.

Successful initiatives are often based on key individuals’ competencies rather than o ganisation wide capability and past successes cannot be repeated consistently.

Processes are undeveloped or incomplete. There is little or no guidance or supporting documentation and even terminology may not be standardised.

Maturity Level 2 - Repeatable Processes

Basic management practices, e.g. tracking expenditure and scheduling resources, are in place and being improved. Key individuals are trained and demonstrate a successful track record and through them, the organisation is capable of repeating success.

Initiatives are performed and managed according to their documented plans; project status and delivery is visible to management at defined points.

There may still be inadequate measures of success; unclear responsibilities; Ambiguity /inconsistency in business objectives; unintegrated Risk Management; limited Change Management; and inadequacies in communications strategy.

Maturity Level 3 – Defined Processes

Management and technical processes are documented, standardised and integrated to some extent with business processes. There is some pr cess ownership and a group responsible for maintaining consistency and delivering process improvements

Senior management are ngaged consistently, providing active and informed support.

There is an established training programme to develop individual’s skills and knowledge.

Maturity Level 4 – Mana ed P ocesses

The organisation has defined processes that are quantitatively managed, i.e. controlled using metrics. There are quantitative objectives for quality and process performance, and these are being used in managing processes.

Top management are proactively seeking out innovative ways to achieve goals.

Using m trics, management can effectively control processes and identify ways to adjust and adapt them to part cular initiatives without loss of quality.

Maturity Level 5 – Optimised Processes

There is focus on optimisation of quantitatively managed processes to account for changing business needs. The organisation exhibits continuous process improvement, and can show strong alignment of organisational objectives with business plans.

Top managers are seen as exemplars, reinforcing the need and potential for capability and performance improvement.

Information from process and product metrics enables the organisation to understand causes of variation and to optimise its performance.

Releas

ed by

ACIC

unde

r FOI


Government




www.oakton.com.au

ATTACHMENT B - TERMINOLOGY

Acronym CRIMTRAC or P3M3™ / OGC Term

P3M3 Portfolio, Programme, Project Management Maturity Model

ACO Accredited Consulting Organisation

RC Registered Consultant

APMG The organisation that maintains certifications for ACOs and RCs.

BoM Board of Management

SIG Strategic Issues Group

PMO Project Management Office

Releas

ed by

ACIC

unde

r FOI

Documents

by ACIC under FOI · As the P3M3 requires that the lowest of the scores sets the maturity level for a sub model, the maturity assessment for Programme Management is reduced to one