Assessing Corporate IT Governance with CMMI and MCDM

Ying-Hsun Hung 1, Wen-Kuo Chen2, Gwo-Hshiung Tzeng3

1Department of Management Information System, Hwa-Hsia Institute of Technology

2Department of Marketing and Logistics Management, Chaoyang University of Technology

3National Distinguished Chair Professor, Institute of Technology Management, National Chiao Tung University

sean@cc.hwh.edu.tw1, wkchen@cyut.edu.tw

2 ghtzeng@cc.nctu.edu.tw

3

Information Technology (IT) has been a focus of research and practice for many decades.

Many organizations have implemented IT governance frameworks to improve their

management and governance of IT. ITIL (Information Technology Infrastructure Library) is

one of widely accepted approaches to IT service management. It is one of frameworks

designed to assist firms by providing them with consistent and comprehensive documentation

of best practice drawn from the public and private sectors internationally. However,

practitioners are often puzzled about where they stand, how well they are doing, and what

they should do next. Many researchers conduct Capability Maturity Model Integration

(CMMI) as a process improvement approach to help organizations to improve their

performance.

Managers usually make strategic decisions based on a single purpose or dimension, but

strategic planning is influenced by many different factors and viewed from several

perspectives, such as cultural, technological and structural standpoints. DANP

(DEMATEL-based Analytic Network Process) is a new multiple criteria decision making

approach used to gathers collective knowledge to capture the causal relationships between

strategic criteria. Therefore, this paper proposes a hybrid assessment approach combined by

CMMI and DANP as a means to help answer these and related questions.

Keywords: CMMI, DANP, ITIL, IT governance,QFD,MCDM

**Wen-Kuo Chen ( wkchen@cyut.edu.tw) is the corresponding author.

1. Introduction

Information Technology (IT) has been a focus of research and practice for many decades.

Many organizations have implemented IT governance frameworks to improve their

management and governance of IT. ITIL (Information Technology Infrastructure Library) is

one of widely accepted approaches to IT service management. It is one of frameworks

designed to assist firms by providing them with consistent and comprehensive documentation

of best practice drawn from the public and private sectors internationally. However,

practitioners are often puzzled about where they stand, how well they are doing, and what

they should do next. Many researchers conduct Capability Maturity Model Integration

(CMMI) as a process improvement approach to help organizations to improve their

performance.

Managers usually make strategic decisions based on a single purpose or dimension, but

strategic planning is influenced by many different factors and viewed from several

perspectives, such as cultural, technological and structural standpoints. DANP

(DEMATEL-based Analytic Network Process) is a new multiple criteria decision making

approach used to gathers collective knowledge to capture the causal relationships between

strategic criteria. Therefore, this paper proposes a hybrid assessment approach combined by

CMMI and DANP as a means to help answer these and related questions.

2. IT service management.

The Information Technology Infrastructure Library (ITIL) is a set of concepts and practices

for Information Technology Services Management (ITSM), Information Technology (IT)

development and IT operations. ITIL gives detailed descriptions of a number of important IT

practices and provides comprehensive checklists, tasks and procedures that any IT

organization can tailor to its needs. The IT Infrastructure Library originated as a collection of

books, each covering a specific practice within IT Service Management. ITIL was built

around a process-model based view of controlling and managing operations often credited

to W. Edwards Deming and his plan-do-check-act (PDCA) cycle.

2.1 Overview of the ITIL v3 library

ITIL v3 is an extension of ITIL v2 and does not replace it. The two publications

should therefore not be considered in isolation. ITIL v3 provides a more holistic

perspective on the full life cycle of services, covering the entire IT organisation and

all supporting components needed to deliver services to the customer, whereas v2

focused on specific activities directly related to service delivery and support. Most of

the v2 activities remained untouched in v3, but some significant changes in

terminology were introduced in order to facilitate the expansion.

Five volumes comprise the ITIL v3, published in May 2007:

1. ITIL Service Strategy

2. ITIL Service Design

3. ITIL Service Transition

4. ITIL Service Operation

5. ITIL Continual Service Improvement

2.1.1 ITIL Service Strategy

As the center and origin point of the ITIL Service Lifecycle, the ITIL Service Strategy

volume provides guidance on clarification and prioritization of service-provider

investments in services. More generally, Service Strategy focuses on helping IT

organizations improve and develop over the long term. In both cases, Service Strategy

relies largely upon a market-driven approach. Key topics covered include service

value definition, business-case development, service assets, market analysis, and

service provider types. List of covered processes:

(1). Service Portfolio Management

(2). Demand Management

(3). IT Financial Management

2.2.2 ITIL Service Design

The ITIL Service Design volume provides good-practice guidance on the design of IT

services, processes, and other aspects of the service management effort. Significantly,

design within ITIL is understood to encompass all elements relevant to technology

service delivery, rather than focusing solely on design of the technology itself. As

such, Service Design addresses how a planned service solution interacts with the

larger business and technical environments, service management systems required to

support the service, processes which interact with the service, technology, and

architecture required to support the service, and the supply chain required to support

the planned service. Within ITIL v2, design work for an IT service is aggregated into

a single Service Design Package (SDP). Service Design Packages, along with other

information about services, are managed within the service catalogues. List of

covered processes:

(1). Service Catalogue Management

(2). Service Level Management

(3). Risk Management

(4). Capacity Management

(5). Availability Management

(6). IT Service Continuity Management

(7). Information Security Management

(8). Compliance Management

(9). IT Architecture Management

(10). Supplier Management

2.2.3 ITIL Service Transition

Service transition, as described by the ITIL Service Transition volume, relates to the

delivery of services required by a business into live/operational use, and often

encompasses the "project" side of IT rather than "BAU" (Business as usual). This area

also covers topics such as managing changes to the "BAU" environment.

List of processes:

(1). Service Asset and Configuration Management

(2). Service Validation and Testing

(3). Evaluation

(4). Release Management

(5). Change Management

(6). Knowledge Management

2.2.4 ITIL Service Operation

Best practice for achieving the delivery of agreed levels of services both to end-users

and the customers (where "customers" refer to those individuals who pay for the

service and negotiate the SLAs). Service operation, as described in the ITIL Service

Operation volume, is the part of the lifecycle where the services and value is actually

directly delivered. Also the monitoring of problems and balance between service

reliability and cost etc. are considered. The functions include technical management,

application management, operations management and Service Desk as well as,

responsibilities for staff engaging in Service Operation.

List of processes:

(1). Event Management

(2). Incident Management

(3). Problem Management

(4). Request Fulfillment

(5). Access Management

2.2.5 ITIL Continual Service Improvement (CSI)

Aligning and realigning IT services to changing business needs (because standstill

implies decline). Continual Service Improvement, defined in the ITIL Continual

Service Improvement volume, aims to align and realign IT Services to changing

business needs by identifying and implementing improvements to the IT services that

support the Business Processes. The perspective of CSI on improvement is the

business perspective of service quality, even though CSI aims to improve process

effectiveness, efficiency and cost effectiveness of the IT processes through the whole

lifecycle. To manage improvement, CSI should clearly define what should be

controlled and measured.

CSI needs to be treated just like any other service practice. There needs to be upfront

planning, training and awareness, ongoing scheduling, roles created, ownership

assigned, and activities identified to be successful. CSI must be planned and

scheduled as process with defined activities, inputs, outputs, roles and reporting.

List of processes:

(1). Service Level Management

(2). Service Measurement and Reporting

(3). Continual Service Improvement

Figure 1. ITIL V.3 Framework

Source: ITIL V3 Foundation

2.2.6 Information Technology Services Assessment in Organizations

Recently many researchers evaluated the perception gaps of service quality between

information technology service providers and their clients to assess these gaps by using the

instrument SERVPERF of the SERVQUAL model.

E-Government services such as the online tax filing and payment system (OTFPS) were

assessed by using a theoretical model based on the theory of planned behavior (TPB),

technology adoption model (TAM), diffusion of innovation theory (DOI) to identify the

determinants for acceptance, and the causal relationships among the variables of acceptance

behavior. Furthermore, some researchers discovered e-Government adoption behavior differs

based on service maturity levels, i.e., when functional characteristics of organizational,

technological, economical, and social perspectives of e-Government differ from the

perspectives of service maturity stages.

3. DEMATEL-based Analytic Network Process (DANP)

3.1 The DEMATEL Method

Because evaluation of knowledge management capabilities cannot accurately

estimate each considered criterion in terms of numerical values for the alternatives,

fuzziness is an appropriate approach. The DEMATEL method is an emerging method

that gathers group knowledge to capture the causal relationships between criteria.

The original DEMATEL (DEcision-MAking Trial and Evaluation Laboratory)

method studied the disjointed and antagonistic phenomena of world and investigated

integrated solutions. In 1973, the Battelle Memorial Institute conducted the

DEMATEL project through its Geneva Research Centre. In recent years, this method

has become very popular in Japan.

It is especially practical and useful for visualizing the structure of complicated

causal relationships with matrices or digraphs, which portray the contextual relations

between the elements of a system, where a numeral represents the strength of

influence. Therefore, the DEMATEL method can convert the relationship between the

causes and effects of criteria into an intelligible structural model of the system.

The DEMATEL method has been successfully applied in many fields. For

example, Tamura et al. (2002) try to decrease anxiety of people by extracting and

analyzing various uneasy factors in order to create future safe, secure and reliable

(SSR) society. More recently, Chiu et al. (2005) adopted the method to study

marketing strategy based on customer behavior related to LCD-TVs. Also Hori and

Shimizu (1999) employed it to design and evaluate the software of a display-screen

structure for analyzing a supervisory control system.

3.2 The DEMATEL for Constructing a NRM

The DEMATEL method (Gabus and Fontela 1972; Ou Yang et al. 2008) was

utilized to investigate the interrelations among criteria to build a NRM. The technique

has been successfully applied in many situations, such as development strategies,

management systems, e-learning evaluations, and knowledge management (Lin and

Tzeng 2009; Tsai and Chou 2009; Tzeng et al. 2007; Wu 2008). The method can be

arranged as follows:

Step 1: Obtain the direct-influence matrix by scores. Respondents are required to

point out the degree of direct influence among each criterion. We suppose that the

comparison scales, 0, 1, 2, 3 and 4, stand for the levels from “no influence” to “very

high influence”. Then, the graph which can describe the interrelationships between the

criteria of the system is shown in Fig. 1. For instance, an arrow from w to y

symbolizes that w impacts on y, and the score of influence is 1. The direct-influence

matrix, A, can be derived by indicated one criterion i impact on another criterion j as

aij.

11 1 1

1

1

j n

i ij in

n nj nn

a a a

a a a

a a a

A

Step 2: Calculate the normalized direct-influence matrix S. S can be calculated by

normalizing A through Equations (6-1) and (6-2).

=m S A (6-1)

1 1

1 1min ,

max | | max | |n n

ij iji j

j i

ma a

(6-2)

Step 3: Derive the total direct-influence matrix T. T of NRM can be derived by using

a formula (6-3), where I denotes the identity matrix; i.e., a continuous decrease of the indirect effects of problems along the powers of ,S e.g.,

2 3, ,..., qS S S and lim [0] ,qn n

q

S where [ ]ij n ns S , 0 1ijs and

0 i ijs or 1j ij

s only one column or one row sum equals 1, but not all.

The total-influence matrix is listed as follows.

T 2 q S S S 2 1 1( )( )( ) qS I S S S I S I S

-1( )( )q S I S I S

when ,q qn n S , then

1( )I T S S (6-3)

where [ ]ij n nt T , , 1, 2,..., .i j n

Step 4: Construct the NRM based on the vectors r and c. The vectors r and c of matrix

T represent the sums of rows and columns respectively, which are shown as

Equations (6-4) and (6-5).

11 1

[ ]n

i n ijj n

r t

r (6-4)

11 1

[ ]n

j n iji n

d t

d (6-5)

where ir denotes the sum of the i th row of matrix T and displays the sum of

direct and indirect effects of criterion i on another criteria. Also, jd denotes

the sum of the j th column of matrix T and represents the sum of direct and

indirect effects that criterion j has received from another criteria. Moreover,

when i j ( )i ir d , it presents the index of the degree of influences given

and received; i.e., ( )i ir d reveals the strength of the central role that factor i

plays in the problem. If ( )i ir d is positive representing that other factors are

impacted by factor i . On the contrary, if ( )i ir d is negative, other factors

has influences on factor i and thus the NRM can be constructed (Liou et al.

2007; Tzeng et al. 2007).

4. Capability Maturity Model Integration (CMMI)

In 1991, the Software Engineering Institute (SEI) of Carnegie Mellon University

introduced the capability maturity model for software (SW-CMM) to evaluate the

capability maturity of software development contractors of the US Defense

Department and provide a roadmap for software process improvement. Since SEI

released SW-CMM Version 1.1, it has been applied to different areas. Hence, many

capability maturity models have been proposed, including the software acquisition

CMM (SA-CMM), system engineering CMM (SE-CMM), integrated product

development CMM (IPD-CMM) and people CMM (P-CMM)(Huang and Han

2006).

Table 1. CMMI : SW-CMM Version 1.1

Source: SW-CMM Version 1.1

As these models were developed by different organizations, they had many

overlapping applications and lacked consistency in architecture, terminology, and

assessment methodology. These problems increased costs and the time required to

implement multiple model-based process improvements. Therefore, SEI released the

capability maturity model integration (CMMI) system in 2001 to integrate existing

capability maturity models. Huang and Han (2006) observed that the advantages of

CMMI are: (1) it eliminates inconsistencies and duplication, and thus streamlines

enterprise-wide process improvements; and (2) it reduces the cost and time associated

with model-based process improvement, and thereby increases the return on an

organization’s investments.

To accommodate different process-improvement needs for software organizations,

the CMMI product team provided them with two choices (staged or continuous

representation) to increase the maturity of their processes. Under CMM, staged

representation provides a framework for organizing the evolutionary steps into five

levels of maturity (initial, managed, defined, quantitatively managed, and optimizing).

(Huang and Han 2006)

These levels are ordinal scales for measuring the maturity of an organization’s

software process and can also be used for its internal process improvement. SEI then

added the continuous representation framework to the CMMI. Because of its

flexibility, the framework enables software organizations to choose their improvement

paths. The continuous representation facilitates comparisons of a specific process area

across software organizations, and thereby allows process improvement to be

compared with that of the ISO/IEC 15504 standard. (Huang and Han 2006)

Source: SW-CMM Version 1.1

Figure 2. Rational of CMMI Staged Representation

5. Quality Function Deployment approach(QFD)

Quality function deployment (QFD) is a valuable method that provides a means of

translating customer needs into the appropriate technical requirements for each stage

of detailed operations in product development and production.

The purpose of this study is to develop an effective decision-making method

based on QFD and DANP approach to help for making better decisions of planning or

evaluation problems.

The QFD is an integrated planning method that can assure and improve the

alignment of elements of design processes with the requirements of customers, as well

as it is a managerial philosophy that can help enhance the organizational and

managing effects [40]. Especially, QFD employs a cross-functional team to plan and

design new or improved products or services through a structured and

well-documented framework. In contrast with traditional requirements of engineering

methodologies, benefits of using QFD are such as: carries the voice of the customer

into the process; abolishes waste and creates flexibility; supports customer-oriented

decisions of design; determines objectives and creates focus on the essential; takes

interests of various groups into account; systematizes communication and provides for

continuity and responsiveness; creates transparency and makes coordination processes

easier; and speeds up development process.

5.1 What QFD can do

According to Wikipedia’s definition, Quality function deployment (QFD) is a

“method to transform user demands into design quality, to deploy the functions

forming quality, and to deploy methods for achieving the design quality into

subsystems and component parts, and ultimately to specific elements of the

manufacturing process.” , as described by Dr. Yoji Akao, who originally developed

QFD in Japan in 1966, when the author combined his work in quality

assurance and quality control points with function deployment used in value

engineering.

QFD is especially designed to help practitioners focus on characteristics of a new

or existing product or service from the viewpoints of market segments, company, or

technology-development needs. The technique yields graphs and matrices.

QFD helps transform customer needs ( the voice of the customer [VOC] )

into engineering characteristics (and appropriate test methods) for a product or service,

prioritizing each product or service characteristic while simultaneously setting

development targets for product or service.

5.2 House of Quality

House of Quality is a diagram, resembling a house, used for defining the

relationship between customer desires and the firm/product capabilities. It is a part of

the Quality Function Deployment (QFD) and it utilizes a planning matrix to relate

what the customer wants to how a firm (that produces the products) is going to meet

those wants. It looks like a House with a "correlation matrix" as its roof, customer

wants versus product features as the main part, competitor evaluation as the porch etc.

It is based on "the belief that products should be designed to reflect customers' desires

and tastes". It also is reported to increase cross functional integration within

organizations using it, especially between marketing, engineering and manufacturing.

The basic structure is a table with "Whats" as the labels on the left and "Hows"

across the top. The roof is a diagonal matrix of "Hows vs. Hows" and the body of the

house is a matrix of "Whats vs. Hows". Both of these matrices are filled with

indicators of whether the interaction of the specific item is a strong positive, a strong

negative, or somewhere in between. Additional annexes on the right side and bottom

hold the "Whys" (market research, etc.) and the "How Muches". Rankings based on

the Whys and the correlations can be used to calculate priorities for the Hows.

House of Quality analysis can also be cascaded, with "Hows" from one level

becoming the "Whats" of a lower level; as these progresses the decisions get closer to

the engineering/manufacturing details.

Moreover, QFD has been applied in various industries such as transportation,

communication, electronics, electrical utilities, software systems, manufacturing,

services, education, and research. It has been successfully applied in many companies

as a powerful tool that addresses strategic and operational decisions in businesses.

To advance continuous improvement for strengthening global competitiveness,

most companies are striving to eliminate the IT services gaps and seize users’ needs

and to seek for higher levels of quality for their products and services. To fulfill these

IT service demands, we adopt Quality Function Deployment (QFD) with ITIL to

provide profitable solutions with the emphasis on goal-oriented, fast, flexible and

customer-focused approach.

Figure 3. QFD House of Quality for Enterprise Product Development Processes

6. A novel hybrid ITIL service approach combined by DANP and QFD

The aim of this study is to build in the service user satisfactions at the IT system

services phase, which is attracting a lot of attention in recent researches.

During the QFD implement process, the Analytic Network Process (ANP) has

been used to determine the relative importance weights between criteria or the

intensity of the relationship between the row and column variables of each matrix.

To develop an effective decision-making method to help for making better

decisions of planning or evaluation problems.

Since the key tool of QFD is the matrix, we focus on the series of interactive

matrices and therefore apply the super-matrix of the ANP in order to perform our

proposed method. In the proposed method, it incorporates several QFD matrices into a

super-matrix based on the Series System model. The procedures of proposed method

are mainly divided into two phases as follows.

Phase 1: Using QFD to develop decision structure.

In this phase, it begins with the way to confirm strategic needs obtained through

business surveys and analyses. Next, it is necessary to define the decision goal and to

collect relevant information, evaluation criteria, and the alternatives. Then, these

decision elements are structured into a three dimension HOQ through the QFD

methodology. Commonly, persons employ the way of traditional two-dimension HOQ,

so that they need to use two HOQs with twice translations. Where the first HOQ

translates the criteria into sub-criteria, and then the second HOQ converts the

sub-criteria into the alternatives. Obviously, the way of using a three-dimension HOQ

is more effective and beneficial than that way of using a two-dimension HOQ.

Because the former provides more integrated information in a compact form, and it is

also more convenient for the calculations with the super-matrix of the ANP. Of course,

if necessary, a four dimension HOQ can also be extended to use.

Phase 2: Using ANP to prioritize alternative.

Once that decision structure is settled down, it is required to employ Saaty’s

five-point scale for making all paired comparisons of decision elements, and then to

incorporate all sub-matrices into the super-matrix which is a hierarchy structure with

four levels including inner dependences. Through performing calculations with the

super-matrix of the ANP, finally the overall priorities of alternatives may be obtained.

As for the calculations of the super-matrix, we can easily solve it with the ways using

either the Microsoft Excel, Mathlab, or the professional software named “Super

Decisions” provided by the Creative Decisions Foundation.

7. Reference

[1]. Y.Q. Yang, S.Q. Wang, M. Dulaimi, S.P. Low, “A fuzzy quality function deployment system

for buildable design decision-makings,” Automation in Construction Vol.12, No.4, 2003,

pp.381-393.

[2]. Akao, Y., Inayashi, K., (1999), “A Study of Service and Operational Quality-An Application

of QFD in Library Services (Keynote),” Proceedings of 5th ISQFD. Brazil. 25th August 1999

pp.200-211.

[3]. Akao, Y. (2002),“QFD and Knowledge Management＂ (keynote), Proceedings of 7th

International Conference on ISO9000 and TQM (ICIT), Melbourne, Australia.

[4]. House of Quality , Wikipedia, http://en.wikipedia.org/wiki/House_of_Quality

[5]. Quality function deployment, Wikipedia,

http://en.wikipedia.org/wiki/Quality_function_deployment

[6]. Prof. Yoji Akao & Kei Inayoshi, “QFD and Administrative Knowledge Management,” Going

for Gold ~ Tutorial on QFD ,12-ICIT, 11/4/2007

[7]. YU-TING LEE, WEI-WEN WU, GWO-HSHIUNG TZENG , “An effective decision-making

method using a combined QFD and ANP approach,” WSEAS TRANSACTIONS on

BUSINESS and ECONOMICS, (12, 5), December 2008.

[8]. Hung, S.-Y., C.-M. Chang, et al. (2006). "Determinants of user acceptance of the

e-Government services: The case of online tax filing and payment system." Government

Information Quarterly 23(1): 97-122.

[9]. Hung, S.-Y., K.-Z. Tang, et al. (2009). "User acceptance of intergovernmental services: An

example of electronic document management system." Government Information Quarterly

26(2): 387-397.

[10]. Boone, T. and R. Ganeshan (2001). "The effect of information technology on learning in

professional service organizations." Journal of Operations Management 19(4): 485-495.

[11]. Kang, H. and G. Bradley (2002). "Measuring the performance of IT services: An assessment

of SERVQUAL." International Journal of Accounting Information Systems 3(3): 151-164.

[12]. Lee, J.-W. (2010). "Online support service quality, online learning acceptance, and student

satisfaction." The Internet and Higher Education 13(4): 277-283.

[13]. McNaughton, B., P. Ray, et al. (2010). "Designing an evaluation framework for IT service

management." Information & Management 47(4): 219-225.

[14]. Roses, L. K., N. Hoppen, et al. (2009). "Management of perceptions of information

technology service quality." Journal of Business Research 62(9): 876-882.

[15]. Shareef, M. A., V. Kumar, et al. (2011). "e-Government Adoption Model (GAM): Differing

service maturity levels." Government Information Quarterly 28(1): 17-35.