Mitigating Technology Implementation Risks: Embedding Software Engineering and Controls Practices into the Organization’s Project Structure

Joseph Robinson

ETM 5121

July 7, 2004

Catalyst for Change

The Enron and WorldCom corporate scandals have forever changed the way organizations will conduct business operations

These events have provided the catalyst for the creation of government regulations: Sarbanes-Oxley Bill (named after the two members of the U.S. Congress and Senate who were instrumental in the bill’s passing)

Sarbanes-Oxley requires that CEO’s and CFO’s attest to the integrity and accuracy of the particular organization’s financial reporting

At the core of this integrity lies a company’s information systems: the source, repository, and reporting mechanism of financial data for most publicly held organizations

This forthcoming proposal seeks to mitigate, or at best minimize, the risk of newly implemented information systems’ data integrity

Proposal for an Embedded Risk Mitigation Structure

It is proposed that the organization utilize certain key processes and steps that will be embedded within the current project structure

The employment of this embedded risk mitigation structure will enhance the current project process, and more importantly the technology implementation, in several ways:

1) Provide clear guidance to all project participants around the mitigation of data integrity risks

2) Outline the risk mitigation process to be employed

3) Ensure risk mitigation activities are formalized

4) Ensure the completeness of these activities through the identification of specific project milestones

Ultimately, this risk mitigation structure will provide assurance that the organization is applying due diligence in addressing the accuracy and integrity of its financial data

Lessons from Recent History

During the phased implementation of the company’s newly purchased point- of-sale system, certain problems surfaced:

1) Transactions causing a random printer freeze condition

2) Transactions dropped from the system record without warning

3) Customers’ credit card purchases captured incorrectly

Although these software and hardware problems were purportedly identified and acknowledged during the testing phase of the implementation project, a decision to continue with the scaled rollout was determined

The risk posed to financial reporting: Missing and incorrect transactions aggregated over the total population of retail locations, some 4,000 stores, could have a substantial impact on the company’s reported sales

These failed transactions also require human effort to mitigate: As the organization continues to ramp up this new technology, the manual effort will also increase

A Mandate for Expanded Project Scope

Proposal: Expand scope of the current project structure to include the minimum activities, or components, necessary to support a technology risk mitigation structure

Justification: 1) Ensures the level of risk to financial reporting inaccuracy is minimized through a formal and repeatable process and 2) aligns with the company’s current Sarbanes-Oxley compliance efforts: documentation of business processes, identification of the control structure1 and any control gaps, and testing of the control environment2

Potential cost of doing nothing: The absence of applied structure creates a potential lapse in the assessment of impending implementations, potentially leading to a condition wherein the accuracy of financial reporting is at risk

[1] A control structure refers to the manual (human process) and automated (system process) controls that provide a level of assurance that risk is minimized or prevented. [2] The control environment consists of a series of manual and/or automated controls that, combined, provide a level of assurance that risk is minimized or prevented. This can also be referred to as a “web of controls”.

Measures of the Problem

The new technology risk mitigation structure’s measures will directly support the major objectives of the initiative: 1) Provide clear guidance to all project participants around the mitigation of data integrity risks 2) Outline the risk mitigation process to be employed 3) Ensure risk mitigation activities are formalized and 4) Ensure the completeness of these activities through specific project milestones

1. Increased understanding of Sarbanes-Oxley requirements and the relationship to technology; measured via surveys conducted through the company’s intranet, prior to and following the proposed program’s rollout

2. Reduction in the number of post-implementation problem tickets opened for the specific technology (vs. past implementations). This measure will be accomplished by comparing the incidence (frequency) and severity of problem tickets relating specifically to reporting and the accuracy of data

3. Increased management satisfaction with level of rigor related to new technology risk mitigation; measured via surveys following technology implementations

Project Objectives

The objective of this project is to develop a formalized and repeatable risk mitigation structure, which:

1. Is embedded in the organization’s existing project structure;

2. Provides clear guidance to all project participants around the mitigation of data integrity risks;

3. Outlines the risk mitigation process to be employed;

4. Ensures risk mitigation activities are formalized;

5. And ensures the completeness of these activities through the identification of specific project milestones.

Specific Deliverables

The specific deliverables that address the problem statement and achieve the objectives stated for developing a formalized and repeatable risk mitigation structure are:

1. A revised project template, including Project Charter, Standard MS Project Template, and Test Plan Template that includes specific risk mitigation activities, objectives, requirements, and milestones. [Supports Project Objectives 1, 2, 3, 4, and 5]

2. A formalized process, including a Process Flow Document, for risk assessment activities, including pre-implementation, project phase, and post-implementation process steps. [Supports Project Objectives 3, 4, and 5]

3. A formal communication document, that includes a summary of the Sarbanes-Oxley impact, requirements resulting from this impact, and other relevant information. [Supports Project Objective 2]

4. A global Sarbanes-Oxley calendar that outlines the timeline for certification and the annual recertification (sustainability) timeline and the key activities required supporting these milestones. [Supports Project Objectives 2, 4, and 5]

5. Summary of measures (scorecard), including the surveys and problem ticket (pre- and post- program) analysis. [Supports Project Objectives 1 and 5]

Alternatives to be considered

These alternatives involve both the method of achieving the objectives and ownership and accountability for the program itself; and in some cases, the alternative is simply the specific combination of these variables

1. Risk mitigation processes owned and administered by the business (100% business accountability)

2. Risk mitigation processes owned and administered by IT (100% IT accountability)

3. Risk mitigation processes owned by IT and administered by functional (business) areas (50% IT accountability; 50% business accountability)

Criteria for Evaluating Alternatives

Assess feasibility of each alternative, based on capabilities and limitations and rank these according to a high, medium, or low value. A decision matrix will be used for this process, utilizing specific decision criteria that are then weighted according to importance

Note: Example data only

Project Approach & Structure

Because this particular proposal involves the improvement and enhancement of an existing process, the initial structure employed to support the definition, measurement, and analysis will be Six Sigma’s DMAIC (Define, Measure, Analyze, Improve, and Control) Cycle

Assuming approval of this proposal (and for the purposes of the assignment), a project approach, utilizing the requirements, design, construction, implementation/deployment, and post-implementation review phases will be employed for the “Improve” portion of DMAIC

The sponsor for this project will be the Sarbanes-Oxley Steering Committee; this executive group represents an adequate cross section of all area that will likely be impacted by this proposed project

The author will execute and directly support the project, including defining project requirements (in collaboration with business partners), scope, measurement, analysis, the presentation of findings to the Steering Committee, and management of the implementation, including communications

MSETM Project Plan

Company Project Plan (Define, Measure, and Analyze Phases)

Company Project Plan (Improve, Control, & PIR Phases)

Planned Use of ETM Materials

Management of Technology Implementations: Much of the MSETM coursework provided insight to the challenges and obstacles in implementing technology and process change

Currently, the IEM 5010 course, Leading and Managing Technology Implementation, has provided key insights to issues that were never considered: probity in development and engineering projects, the lessons that failures provide, etc.

Strategic Quality Leadership/Management: Much of the impact of misguided technology implementations is unknown and unknowable. This paraphrased Deming principle is definitely applicable to the point-of-sale example presented earlier in this proposal document

Although each of the MSETM courses has provided, and continue to provide, key insights and learning, the courses outlined below are especially significant in terms of this project proposal:

References

1. Breyfogle, F., Cupello, J, and Meadows, B. Managing Six Sigma: A practical guide to understanding, assessing, and implementing the strategy that yields bottom-line success. New York, NY: John Wiley & Sons, Inc., 2001. 

2. Squires, A.M. The Tender Ship: Governmental Management of Technological Change. Cambridge, MA: Birkhauser Boston, Inc., 1986.