1 | P a g e

Determining Key Attributes for Profiling the

Information Technology (IT) Portfolios

June 22, 2015

[Anonymized for review]

[Note: A graduate student has done the bulk of this work.]

Abstract

Prior studies have shown that the key concept of IT Portfolio Management (ITPM) is to improve

the performance of IT investment and to optimize business value for the entire enterprise. To

exploit the ITPM domain, we aim to integrate critical IT attributes with enterprise strategic goals

to address the fundamental concept of IT investment planning and decision-making in this paper.

Through incorporating the chosen IT portfolio attributes using Data Envelopment Analysis

(DEA)-based model, a firm will be able to optimize efficiency of organizational units driven by

IT portfolios and to systematically profile numerous IT portfolios via efficient frontiers, also

known as best-practice frontiers, to better articulate upcoming IT investment decisions.

Furthermore, since the conventional DEA model only measures efficiency at a sole

organizational level, we extend toward a new ITPM model, termed the DEA/Parallel (DEA/P)

model, to optimize efficiency across organizational levels in a multi-business unit firm. Our

methodology incorporates mathematical optimization and computational experiments, along with

combining real-world data using the Monte Carlo approach, to simulate a firm’s IT portfolios

and provide theoretical insights into the components of the optimal solution. Based on the

analysis of using two DEA-based models (conventional DEA model and the proposed DEA/P

2 | P a g e

model), we compare the differences of these two models through three aspects: efficiency score,

efficient frontier, and distance function. Specifically, instead of being determined by a senior

executive’s intuition, weight scores produced by our proposed DEA/P model enable a firm to

create a rationale viewpoint of how much to invest in each strategic goal to improve investment

efficiency. Accordingly, our findings indicate that the conventional DEA model’s efficient

frontier could potentially be applied to a lower bound for investing in less risky business

objectives, while the DEA/P model’s efficient frontier could be applied to a higher bound for

managing IT resources before entering a risky market. Therefore, the two main contributions of

this paper are as follows: 1) a new methodology, which is used to demonstrate the optimality that

measures the efficiency of IT resource allocation driven by IT portfolios across multi-

organizational levels/units simultaneously and 2) strategies for locating efficient frontier and

distance function, which are used to illustrate graphical results profiling the chosen IT portfolio

attributes and to indicate the gap between the best-practice frontier and inefficient Decision

Making Unit (DMU) aimed at the follow-up efficiency improvement.

Keywords: IT Portfolio Management (ITPM), IT portfolio attributes, Data Envelopment Analysis (DEA),

efficient frontier

3 | P a g e

I. Introduction

Typically, information technology (IT) is considered to have a cost saving role among various

business functions in a firm. Chan et al. (1997) found that the “fit” between Information System

(IS) and business objectives is significantly associated with the performance of a firm. According

to the latest forecast by the research firm Gartner, Inc. (2014), global IT spending grew by 3.2 %

to total $3.8 trillion U.S. dollars. Following the implementation of the Sarbanes-Oxley Act, many

enterprise investment decisions are strictly examined; as a result, investment issues have become

a great concern nowadays for many firms. To better position IT in terms of business value

creation, IT Portfolio Management (ITPM) aims to improve the performance of IT investment

and to optimize the business value of enterprise IT. In addition, compared to financial-related

investments, the relationship between inputs (e.g., budgeted cost) and outputs (e.g., expected

return) in the IT investment contexts may not be linear. Due to several specific features including

a non-parametric approach and linear fractional programming model, Data Envelopment

Analysis (DEA) is regarded as an appropriate methodology to cope with non-linear problems

related to IT investment issues to demonstrate IT resource allocation (Tanriverdi and Ruefi,

2004). On the other hand, DEA has been applied to solve multi-attribute decision-making

problems in several areas. Through incorporating multiple key IT portfolio attributes using DEA-

based model, the key motivation of our ITPM research is to assist a firm in systematically

profiling numerous IT portfolios that comprise the chosen IT portfolio attributes to better

accomplish its enterprise business objectives.

Following enterprise IT viewpoints, we consider an IT portfolio to be a production unit when

measuring the efficiency of IT resource allocation. Given the organizational benefits of IT

portfolios, making appropriate IT investment decisions to achieve optimal IT resource allocation

4 | P a g e

across different organizational levels has been recognized as one of the critical issues for

enterprise executives. Thus, our research question is: “How can a Multi-Business Unit Firm

Incorporate Numerous IT Portfolio Attributes to Optimize its Efficiency across Organizational

Levels?” To address this research question correspondence to IT resource allocation within the

firm, the two main contributions of this study are as follows: 1) a new methodology, which is

used to demonstrate the optimality by measuring the efficiency of IT resource allocation driven

by IT portfolios across multi-organizational levels/units simultaneously and 2) strategies for

determining efficient frontier and distance function, which are used to illustrate graphical results

profiling the chosen IT portfolio attributes and to indicate the gap between the best-practice

frontier and inefficient Decision Making Unit (DMU) aimed at the follow-up efficiency

improvement. This paper is organized as follows: Section II reviews theoretical studies. Our

model is developed in Section III. In Section IV, the proposed methodology is illustrated with a

hypothetical example. Section V concludes the results and presents managerial interpretations.

II. Theoretical Development

2.1 IT Resources and IT Portfolio Management (ITPM)

Zhu and Kramer (2002) and Zhu (2004) point out that the Resource-Based View (RBV) in

Information System (IS) research has been widely applied to interpret how enterprises are able to

produce competitive value from IT assets. Meanwhile, firm performance, including sustainability,

will increase by leveraging IT. According to Melville, Kraemer and Gurbaxani (2004), the RBV

is used to resolve the productivity paradox and to explain how firms create business value in

connection with an organization’s competences. In accordance with Hitt and Brynjoifsson (1996),

the production theory can be used to evaluate IT investments regarding IT productivity.

5 | P a g e

Regarding the enterprise IT point of view, an IT project is the main tactical level through which

IT activity translates to business results for the enterprise. IT project selection is an essential

business problem because most IT components are customized for an enterprise through project

implementation (Cho and Shaw, 2013). Integrating Zhu (2003) and Ray et al. (2005)’s concepts,

an IT portfolio level can be considered a bridge that connects project levels to the firm level

regarding strategic IT resource allocation. The IT portfolio of a firm is understood as its total

investment in computing and communication technology (Weill and Vitale 2002), or the sum

total of all IT projects. According to Jeffery and Leliveld (2004), the definition of IT portfolio

management (ITPM) is to manage IT as a portfolio of assets similar to a financial portfolio and

then strive to improve the performance of the portfolio by balancing risk and return. The key

motivation for doing ITPM is to find the most applicable IT portfolio to improve the

performance of IT investment and to optimize the business value of enterprise IT.

2.2 Productivity Theory and Data Envelopment Analysis (DEA)

Prior research has shown that production theory has been widely utilized to uncover how best to

combine resource inputs to achieve desired outcomes, and production theory can be used to

evaluate IT investments concerning IT productivity (Hitt and Brynjoifsson, 1996). Among

different production functions, one method, namely, Data Envelopment Analysis (DEA),

proposed by Charnes, Cooper and Rhodes (1978), does not require any particular characteristics

such as statistical distribution and functional form. In this respect, the DEA model is broadly

used to estimate productivity analysis to address the inputs consumed by the outputs produced,

and it also shows the tradeoffs in achieving various performance metrics (Banker et al., 2004,

2011). Compared to other approaches, the DEA model lessens the complexity of analysis by

6 | P a g e

concurrently measuring the relevant attributes of multiple Decision Making Units (DMUs) and

then turning out a composite score, referred to as efficiency (Powers and McMullen, 2000).

2.3 Why use DEA to Solve IT Portfolio Management (ITPM) problems?

ITPM involves making applicable decisions to achieve a firm’s strategic objectives by fine-

tuning budgeted costs and returns as business conditions change; thus, the objectives of ITPM

are to plan, measure and optimize the business value of enterprise IT. With reference to financial

economics literature, the relationship between return and risk is positively linear, whereas the

relationship between return and risk regarding IT investments may be non-linear (Tanriverdi and

Ruefli, 2004). Motivated by the non-linear relationship between return and risk in IT investment,

the DEA model is known as a non-parametric approach and a linear fractional programming

model, and there is no need for DEA to include explicit mathematical forms between inputs (e.g.,

risk) and outputs (e.g., return). For these reasons, DEA is an appropriate methodology to uncover

hidden relationships among multiple inputs and outputs while incorporating the chosen IT

portfolio attributes in the ITPM field to demonstrate IT resource allocation. To put it simply,

DEA can be used with heterogeneous metrics of inputs and outputs in the ITPM context (Cho,

2010). Additionally, since both budgeted cost and benefit can be seen as key IT portfolio

attributes related to scarce firm resources most of the time, combining both of them with IT

function enhances IT productivity and, in turn, the organization’s growth. Accordingly, we

summarize the numerous main IT portfolio attributes in the Appendix.

2.4 How can DEA be applied to the IT Portfolio Management (ITPM) context?

Addressing portfolio problems is linked to the challenge of enterprise resource allocation to

maximize value in most cases (Dia, 2009); therefore, there is a need to develop an appropriate

7 | P a g e

method that can combine various IT portfolios attributes to measure the productivity of IT

project portfolios in the ITPM domain. Briefly, IT productivity is regarded as the relationship

between a firm's IT-related investments and its associated efficiency gains, such as financial

returns. To tackle this issue, since IT function-based strategic goals are implemented by a set of

IT-related projects, called an IT (project) portfolio, we can apply DEA-based models to optimize

the relative efficiency of organizational units driven by IT portfolios and thereby demonstrate the

organizational performance. Further, while applying the ITPM context to better accomplish

enterprise business objectives, the DEA-related model is able to not only incorporate multiple IT

portfolio attributes to address IT resource allocation but also to generate the efficient frontier (the

best-practice frontier), which appears as a graphical outcome of optimal combination of inputs

and outputs for a firm systematically profiling numerous IT portfolios.

2.5 Parallel DEA Model and its contributions

Before the concept of parallel production model was applied to the DEA model, researchers

considered either a firm level or an organizational department level to be an individual DMU

without connecting them with lower organizational levels when measuring efficiency of resource

allocation, and, therefore, they did not devise the decomposition of a production system into

several sub-systems. Research on parallel production systems began with Färe and Primont

(1984) building on the conventional DEA model, and Kao (2009) applied the theory and

proposed the general parallel production system with multiple processes operating independently

from earlier work. Moreover, the contribution of the parallel DEA model is to decompose a

system into multiple separated processes through the concept of parallel production system and

then measure both the system and process efficiencies of each DMU in one linear program

framework. Particularly, when applying parallel DEA model to ITPM context built on Kao

8 | P a g e

(2012), senior executives could construct a rationale viewpoint of how much to invest in each

strategic goal, which is implemented by the IT project portfolio, to improve investment

efficiency without the problem of resource duplication.

III. Model Development

3.1 A conventional DEA model for ITPM context

By taking into account the nature and complexity of the relation, the DEA model is a proper

multi-attribute model to estimate inputs (e.g., risk and budgeted cost) and outputs (e.g., expected

returns), while transforming the ratio of multiple inputs and outputs into an equivalent linear

program by a scalar measurement ranging from 0 (the worst) and 1 (the best) for each DMU

(Charnes et al., 1978). Since an IT project is the main level through which IT activity translates

to business results, IT (project) portfolios can be thought of as a pool of heterogeneous IT

projects within a firm. By importing this concept associated with DEA’s features, we will

prioritize the IT projects as DMUs by using the conventional DEA model applied in the ITPM

context as shown in Table 1A, along with variables and definition in Table 1B.

3.2 A new proposed DEA/P model for ITPM context in a multi-business unit firm

Milgrom and Roberts (1990; 1995) assume a one-level firm with resources and activities as its

elements. However, Barua, Lee and Whinston (1996) and Barua and Mukhopadhyay (2000)

contend that the firm should be conceptualized in multiple levels because investments into

resources and activities are converted into firm level performance outcomes through several

intermediate levels. The main reason is that IT plays a role in all levels of the firm, from

investments through intermediate levels to ultimate risk/return performance (Tanriverdi and

Ruefi, 2004). IT portfolios perform the particular IT-related functions not only link to enterprise

9 | P a g e

strategic goals but also to support the associated business plans at each organizational level.

Typically, an enterprise consists of various business units (or organizational departments), and

each business unit may have its own strategic priority to accomplish enterprise IT-driven

strategic goals, which can be realized by a set of ongoing IT projects, known as the IT project

portfolio. For these reasons, we are interested in developing a good mechanism to assess the

productivity of IT resources driven by IT project portfolios to reach IT-driven strategic goals,

while allocating IT resources to different organizational levels in a multi-business unit firm.

Thus, we propose a new ITPM model called the Data Envelopment Analysis /Parallel-based

model, or DEA/P, which builds on the parallel DEA model to optimize efficiency of strategic

goals implemented by various IT portfolios in a parallel resource allocation for a multi-business

firm. Meanwhile, we illustrate the three main components of our proposed DEA/P model in

Figure 1: (1) Business Unit, (2) Strategic Goals implemented by IT Project Portfolio, and (3) IT

Projects. Referring to the DEA/P’s mathematical equations in Table 2A, higher organizational

levels distribute their strategic IT resources into several lower organizational levels via a parallel

approach. In section IV, we will demonstrate our DEA/P model with a hypothetical example

with reference to a Fortune 50 firm’s IT project portfolios. Likewise, how to calculate efficiency

scores for multiple organizational levels by using the DEA/P model can be found in Table 2B.

3.2.1 Parameter/Variable Definition

To address the strategic priority of each business unit or organizational department in a firm, a

weight score, 𝑤(𝐼𝑇𝑃𝑃), derived from the DEA/P model can be regarded as a percentage of IT

resources assigned to an IT (project) portfolio. The weight scores generated by our DEA/P model

is the percentage (%) of IT budget allocation to demonstrate the strategic priority of the

enterprise strategic goal. Therefore, instead of determined by senior executive’s intuitive or

10 | P a g e

subjective experience, the contributions of the DEA/P model’s weight scores enable a firm to

create a rationale viewpoint of how much to invest in each strategic goal to improve investment

efficiency for each business unit. Further, the selection of input and output variables plays an

essential role in the DEA/P model since these variables reflect variations in IT-related resource

utilization across different organizational levels. Along with the DEA/P model components in

Table 2A and Table 2B, we summarize the DEA/P model’s parameters and variables in Table 2C.

3.2.2 Model Assumption and Managerial Interpretation

There are a few assumptions concerning our proposed DEA/P model while measuring the

efficiency of IT resource allocation across multiple business units. Firstly, when a firm assigns

its IT resource to different organizational levels, it will be through a top-down parallel approach

without considering the interdependency value for each organizational level (e.g., business unit,

IT portfolio, and IT project). Secondly, each business unit has its own strategic goals realized by

a number of IT (project) portfolio, and therefore different business units will not share the

assigned IT resources with each other. Thirdly, since the IT Portfolio Management (ITPM)

domain is defined as a continuous process to manage IT project, application, and infrastructure

assets and their interdependencies (Kumar et al., 2008), we will mainly address the IT project

portfolio in this paper. Based on our model assumptions, the summary of managerial

interpretations regarding our proposed DEA/P model in the ITPM context is shown in Table 2D.

IV. Hypothetical Example and Analysis by Using the DEA and DEA/P Models

4.1 Research Design and Simulation Data Description

According to Gartner’s estimate of average IT spending and IT budgeting key initiative overview

(McGittigan, 2014), the average enterprise IT investment is assumed to be 3.5% of revenue and

11 | P a g e

projects are constructed based on an average project size of $4 million. In this study, we depend

on a computational model for simulating firm selection of IT project investment portfolio. Our

model design is based on the research context of this study, which is a US Fortune 50 enterprise

in the finance and insurance sector. The firm’s senior executives review IT project investments

periodically. With respect to research design, we will build a systematic approach to simulate a

large amount of IT project portfolio data based on actual operational parameters from a number

of Fortune 50 firms’ IT project portfolios. To combine the suggested IT portfolio attributes while

addressing ITPM-related issues, our proposed model includes important parameters/variables

that are common but may uniquely characterize the IT portfolio.

Along with the expected skewed distribution, the descriptive statistics for the simulated IT

portfolios are in Table 3 and our simulated data across multiple organizational levels can be

found in Figure 2. Thus, we demonstrate the two DEA models through simulated IT portfolio

data to address 3 business units, 9 IT Portfolios and 90 IT projects.

4.2 Analysis for a simulated multi-business unit firm using the DEA model and DEA/P model

The enterprise IT resources are connected with different organizational levels (i.e. business unit,

IT portfolio, and IT project); therefore, we next evaluate the efficiency of a multi-business unit

firm by using the conventional DEA and our proposed DEA/P model to differentiate these two

DEA-based models through three aspects: (1) Efficiency Score - the efficient score of Decision

Making Unit (DMU) is understood as the status of IT-related resource utilization, (2) Efficient

Frontier - the efficient frontier is the optimal ratio combination between inputs (e.g., budgeted

cost) and outputs (e.g., expected return), and (3) Distance Function associated with Efficient

Frontier – this can illustrate graphical results profiling the chosen IT portfolio attributes and

12 | P a g e

indicate the gap between the best-practice frontier and inefficient DMU aimed at follow-up

efficiency improvement. Detailed discussions related to these three aspects are shown below.

4.2.1 Business Unit Level

(1) Efficiency Score and Efficient Frontier (Best-Practice Frontier)

Three business units are considered to be three DMUs on the subject of the business unit level

associated with on our simulated data, and the efficiency scores (E) produced by the

conventional DEA and our proposed DEA/P model are able to reveal efficiency of IT resource

allocation for three business units shown in Table 4A. Consequently, E = 1 means that an

organizational unit (e.g., business unit) achieves the optimal condition for its IT resource

allocation in comparison to the peer DMUs. Otherwise, the organizational unit would be thought

of as having inefficient status.

In addition, our simulated data in Figure 2 indicates that business unit-B (BU-B) has the lowest

IT investments and business unit-C (BU-C) has the highest IT investments. Based on the

graphical result in Figure 3, two business units (BU-B and BU-C) reach the efficient frontier

produced by the conventional DEA model; however, none of the business units meets the

efficient frontier generated by the DEA/P model. Referring to our results, the senior executives

may leverage the DEA/P model to make more economical IT resources allocation, compared to

the conventional DEA model.

(2) Distance Function associated with Efficient Frontier

Although there are numerous IT portfolio attributes, the efficient frontier in our hypothetical

example is comprised of two input variables (i.e. general spending and labor cost) and one output

variable (expected return). Thus, we found that the business unit-A (BU-A) is the only business

unit that is slightly distant from the efficient frontier. To build on the concept of distance

13 | P a g e

function, the distance between two points of the xy-plane can use the distance formula; therefore,

the distance between (x1, y1) and (x2, y2) is given by: d = √(∆𝑥)2 + (∆𝑦)2

For example, according to the distance formula as shown above, the senior executives could

modify resource allocations to make the BU-A reach the efficient frontier generated by the

conventional DEA model, as the other two business units in Table 4B. Additionally, Table 4B

indicates that three business units have closer distance between the origin and the efficient

frontier produced by the DEA/P model. In line with this perspective, if a firm intends to embrace

an economical investment decision as shown in Figure 3, it may lower both ratios of Labor to

Expected Return and General Spending to Expected Return to reach the efficient frontier

generated by the DEA/P model, ranging from the efficient frontier generated by the conventional

DEA model.

4.2.2 IT (Project) Portfolio Level

(1) Efficiency Score, Weight Score, and Efficient Frontier (Best-Practice Frontier)

In terms of IT-related resource utilization, each business unit may have multiple ongoing IT

portfolios to realize enterprise strategic goals, and we consider IT portfolios to be DMUs in this

section. Compared to the conventional DEA model, our DEA/P model is able to measure the

efficiency of IT portfolios across multiple business units simultaneously and to generate an

additional managerial reference, called a Weight Score, which is defined as how much to invest

in each strategic goal to improve investment efficiency.

In our hypothetical example, both IT portfolio – E (a small-sized IT portfolio focusing on

innovation management in BU-B) and IT portfolio – I (a large-sized IT portfolio focusing on

customer management in BU-C) reach the optimal condition by applying the conventional DEA

model, but these two IT portfolios show there is still some room for efficiency improvement by

14 | P a g e

using DEA/P model. Additionally, the results from the DEA/P model show that these two IT

portfolios belonging to two separated business units reveal very different weight scores (i.e. w =

0.188 for IT portfolio – E and w = 0.71 for IT portfolio – I); therefore, the weight scores can be

used to emphasize their strategic focus as shown in Table 5A. In Figure 4, two IT portfolios (E

and I) reach the efficient frontier produced by the conventional DEA model, but none of the IT

portfolios meets the efficient frontier generated by the DEA/P model. Thus, we mark point E’

and I’ to represent the two optimal points on the efficient frontier generated by the DEA/P model,

and these two points can be seen as targets regarding the upcoming investment decisions.

(2) Distance Function associated with Efficient Frontier

We summarize the distance from origin to each IT portfolio in Table 5B. Since there are only

two IT portfolios that achieved the efficient frontier generated by the DEA model, we further

estimate these two IT portfolios’ distance from origin to the efficient frontier produced by the

DEA/P model. In terms of resource focus, these two IT portfolios may make some changes to

improve their efficiency as follows: IT portfolio - E (a small-sized IT portfolio focusing on

innovation management in BU-B) may need to lower General Spending but increase the Labor

Cost; IT portfolio - I (a large-sized IT portfolio focusing on customer management in BU-C)

may need to reduce the Labor Cost but increase General Spending.

Based on our example, the weight scores from the DEA/P model can be seen a complement

reference for the senior executives to find out the strategic focus, while linking IT resource

allocation to the enterprise strategic goals. As such, a higher weight score can be considered a

more influential strategic focus connected to a certain organizational level. In this regard,

because IT portfolio – I has the highest weight score among others, it will be the first priority to

accomplish a particular strategic goal in our hypothetical example.

15 | P a g e

4.2.3 IT Project Level

(1) Efficiency Score, Weight Score, and Efficient Frontier (Best-Practice Frontier)

The IT portfolio level is considered to be the bridge connecting the firm level to the IT project

level, and IT projects are implemented at a primary level to demonstrate a specific IT-driven

business functions within the enterprise. After analyzing our simulated data as shown in Figure 2,

there are only three IT projects (i.e. A26, A32, and C25) that achieve the optimal condition

among all the 90 IT projects by using the conventional DEA model in Table 6A, whereas there is

not any IT project that appears in the optimal condition by using the DEA/P model in Table 6B.

Also, the efficient frontiers in Figure 5 could be a reference for the senior executives to better IT

resource allocation for the IT projects. Briefly speaking, this shows that the DEA/P model has a

more economic viewpoint concerning the efficiency measurement rather than the conventional

DEA model. Specifically, weight scores produced by the DEA/P model can bring up the

emphasis of resource allocation to better complement the efficiency score.

(2) Distance Function associated with Efficient Frontier

Referring to our analysis along with simulated data, when IT resources are distributed across the

organizational levels (from business unit level to the IT project level), both DEA and DEA/P

model show that the subordinate organizational levels may gain lower efficiency scores due to

the increasing number of production units. For those IT projects that are relatively close to x axis,

we suggest that the senior executives may either reduce the investments in the General Spending

or lift up the Expected Return in order to lower the ratio of General Spending to Expected Return.

On the other hand, for those IT projects that are relatively close to the y axis, we recommend that

the senior executives may either reduce the investments in the Labor Cost or raise the Expected

Return in order to lower the ratio of Labor Cost to Expected Return.

16 | P a g e

V. Discussion

5.1 Conclusion

Along with combining real-world data using the Monte Carlo approach to simulate firms’ IT

portfolios, our methodology incorporates mathematical optimization and computational

experiments to optimize IT resource utilization. Thus, based on the analysis by using two DEA-

based models (conventional DEA model and the proposed DEA/P model), we compare the

differences, including efficiency scores, efficient frontiers, and distance functions. Based on our

hypothetical example, the results indicate that the small-sized IT portfolio focusing on

innovation management and the large-sized IT portfolio focusing on customer management may

easily reach the optimal condition in the customer-oriented business unit by applying the

conventional DEA model, but these two IT portfolios show there is still some room for

efficiency improvement by using DEA/P model. In this regard, the senior executive may better

improve the efficiency of IT resource utilization by using DEA/P model because it can

incorporate both input and output variables across multiple organizational levels that are

connected with each other. Specifically, based on the weight score generated by our proposed

DEA/P model, the “customer management” appears as the most critical strategic goal compared

to others. This could be that our research context of this study is a US Fortune 50 enterprise in

the finance and insurance sector.

5.2 What do the results mean to IT Portfolio Management?

Our findings indicate that the conventional DEA model’s efficient frontier could potentially be

applied to a lower bound for investing in less risky IT portfolios, while the DEA/P model’s

efficient frontier could be applied to a higher bound for managing IT portfolios before entering a

17 | P a g e

risky market. Both DEA and DEA/P model show that the subordinate organizational levels may

gain lower efficiency scores due to the increasing number of production units, and the advantage

of using the DEA/P model over conventional DEA to solve ITPM problems is to have a more

economic viewpoint concerning the efficiency measurement. To address the limitation of the

conventional DEA model, our proposed DEA/P model is able to incorporate all inputs and

outputs related to IT investments across organizational levels that are connected with each other.

Specifically, instead of being determined by a senior executive’s intuition, weight scores

generated by the proposed DEA/P model enable a firm to create a rationale viewpoint of how

much to invest in each strategic goal to improve investment efficiency for the organizational unit.

We aim to assist a firm in comprising the chosen IT portfolio attributes among alternative

choices to better achieve enterprise business objectives when making upcoming investment

decisions. Thus, the two main contributions can be found as follows: 1) a new methodology,

which is used to demonstrate the optimality that measures the efficiency of IT resource allocation

driven by IT portfolios across multi-organizational levels/units simultaneously and 2) strategies

for determining efficient frontier and distance function, which are used to illustrate graphical

results profiling the chosen IT portfolio attributes and to indicate the gap between the best-

practice frontier and inefficient Decision Making Unit (DMU) aimed at the subsequent efficiency

improvement. Regarding our future work, we will run a large-scale simulation to complement

our initial illustrative example, and accordingly, the results from the simulated data may serve as

strong references when applying our proposed ITPM model to better analyze empirical data.

18 | P a g e

Tables

Table 1B – Parameter/Variable and Definition for DEA model

Variable Definition

𝐸𝑗

𝑥1𝑗

The efficiency of IT project j

Estimated cost of IT project j

𝑥2𝑗 Estimated risk of IT project j

𝑦𝑗 Estimated return of IT project j

𝑢 The weight on the return

𝑣1 The weight on the cost

𝑣2 The weight on the risk

Table 1A – DEA model in ITPM context

Max 𝐸𝑗 =𝑢𝑦𝑗

𝑣1𝑥1𝑗 + 𝑣2𝑥2𝑗

Subject to 𝑢𝑦𝑗

𝑣1𝑥1𝑗 + 𝑣2𝑥2𝑗 ≤ 1

j = 1… n

19 | P a g e

Table 2A – DEA/Parallel Model (mathematical equation) in ITPM context

Table 2B – Efficiency Score for multiple organizational levels via DEA/P Model

𝐸𝐵𝑈 = 1 - 𝑠𝑘

(𝐵𝑈)

𝐸𝐼𝑇𝑃𝑃 = 1 - 𝑠𝑘

(𝐼𝑇𝑃𝑃)/ ∑ 𝑣𝑖

𝑚𝑖=𝐼(𝐼𝑇𝑃𝑃) 𝑋𝑖𝑘

(𝐼𝑇𝑃𝑃)

𝐸𝐼𝑇𝑃 = 1 - 𝑠𝑘

(𝐼𝑇𝑃)/ ∑ 𝑣𝑖

𝑚𝑖=𝐼(𝐼𝑇𝑃) 𝑋𝑖𝑘

(𝐼𝑇𝑃)

𝑤(𝐼𝑇𝑃𝑃) = ∑ 𝑣𝑖𝑋𝑖𝑘

(𝐼𝑇𝑃𝑃)i∈𝐼(𝐼𝑇𝑃𝑃)

∑ 𝑣𝑖𝑋𝑖𝑘𝑚𝑖=1

20 | P a g e

Table 2C – Parameter/Variable and Definition for DEA/Parallel Model

Parameter Definition

𝐸𝐵𝑈

𝐸𝐼𝑇𝑃𝑃

𝐸𝐼𝑇𝑃

Efficiency scores of IT-related resource allocation across multi-organizational

levels; that is, firm level, business unit (BU), IT project portfolio (ITPP), and

IT project (ITP), can be generated by our proposed DEA/P model.

v𝑖 Weight on the ith input (IT resource) variable

u𝑟 Weight on the rth output (expected return) variable

𝑋𝑖𝑘(𝐵𝑈)

A certain amount of the ith input (IT resource) is assigned to the specific

Decision Making Unit (DMU) k; therefore, DMU k is considered as a specific

business unit k with regard to the business unit (BU) level.

𝑌𝑟𝑘(𝐵𝑈)

The rth output (expected return) is produced by the specific Decision Making

Unit (DMU) k; therefore, DMU k is considered as a specific business unit k

with regard to the business unit (BU) level.

𝑋𝑖𝑗(𝐵𝑈)

A certain amount of the ith input (IT resource) is assigned to the Decision

Making Unit (DMU) j; therefore, DMU j is considered as a business unit j with

regard to the business unit (BU) level.

𝑌𝑟𝑗(𝐵𝑈)

The rth output (expected return) is produced by the Decision Making Unit

(DMU) j; therefore, DMU j is considered as a business unit j with regard to the

business unit (BU) level.

𝑠𝑘(𝐵𝑈)

The buffer IT resources related to the specific Decision Making Unit (DMU) k;

therefore, DMU k is considered as a specific business unit k with regard to the

business unit (BU) level.

𝑋𝑖𝑘(𝐼𝑇𝑃𝑃)

Amount of input (resource) i required for the IT project portfolio from the

specific Decision Making Unit (DMU) k

𝑌𝑟𝑘(𝐼𝑇𝑃𝑃)

Given certain input-based resource allocations, an amount of output (return) r

expected for the IT project portfolio from the specific Decision Making Unit

(DMU) k

𝑋𝑖𝑗(𝐼𝑇𝑃𝑃)

Amount of input (resource) i required for the IT project portfolio from

Decision Making Unit (DMU) j

𝑌𝑟𝑗(𝐼𝑇𝑃𝑃)

Given certain input-based resource allocation, an amount of output (return) r

expected for the IT project portfolio from Decision Making Unit (DMU) j

𝑠𝑘(𝐼𝑇𝑃𝑃)

The buffer IT resources for the IT project portfolio level under the specific

Decision Making Unit (DMU) k

𝑋𝑖𝑘(𝐼𝑇𝑃)

Amount of input (resource) i required for the IT project from Decision Making

Unit (DMU) k

𝑌𝑟𝑘(𝐼𝑇𝑃)

Given certain input-based resource allocations, an amount of output (return) r

expected for the IT project from Decision Making Unit (DMU) k

21 | P a g e

𝑋𝑖𝑗(𝐼𝑇𝑃)

Amount of input (resource) i required for the IT project from Decision Making

Units (DMU) j

𝑌𝑟𝑗(𝐼𝑇𝑃)

Given certain input-based resource allocations, an amount of output (return) r

expected for the IT project from Decision Making Units (DMU) j

s𝑘(𝐼𝑇𝑃)

The buffer IT resources for the IT project level under the specific Decision

Making Unit (DMU) k

Table 2D – Summary for Parameter/Variable associated with Managerial Interpretation

Parameter/Variable Range Managerial Interpretation

EBU: E-score for Business

Unit or Org. Department

EITPP: E-score for IT

project portfolio level

EITP: E-score for IT

project level

E = 0 (worst) ~ E = 1 (optimal)

The higher efficiency score can be

understood as a better strategic resource

allocation in connection with an

organizational level, as such; E = 1

means the optimal condition for IT-

related strategic resource allocation.

S: Slack score The slack score is associated

with E-score

Utilized resources – a lower score

indicates high utilization and a higher

score indicates organizational slack.

W: Weight score

(strategic option focus) W = 0 (worst) ~ W = 1 (optimal)

The weight score can be seen as how

much to invest in each strategic goal to

improve investment efficiency.

A higher weight score can be considered

a more influential strategic focus

connected to a certain organizational

level.

X1: Input Variable 1

Each hierarchical organizational

level has its amount of resources

related to Labor Cost

Labor Cost

X2: Input Variable 2

Each hierarchical organizational

level has its amount of resources

related to General Spending

General Spending

Y1: Output variable 1

Each hierarchical organizational

level has its amount of resources

related to Expected Return

Expected Return

22 | P a g e

Table 3 – Descriptive Statistics for the Simulated IT Portfolio

Variable Mean Std. Dev.

Budgeted Cost 1 – Labor Cost (X1) $2.96 Million $12.22 Million

Budgeted Cost 2 – General Spending (X2) $8.8 Million $28.72 Million

Expected Return (Y) $12.38 Million $42 Million

Table 4A – Efficiency Score for Business Unit (BU)

DEA model DEA/P model

Business Unit Efficiency score BusinessUnit Efficiency score

BU – A E = 0.969 BU – A E = 0.683

BU – B E = 1 BU – B E = 0.743

BU – C E = 1 BU – C E = 0.728

Table 4B – Distance Function for Business Unit (BU)

DEA model DEA/P model

Business

Unit

Distance from origin

to each business unit

Business

Unit

Distance from origin

to each business unit

BU – A OA = 0.721 BU – A OA’ = 0.537

BU – B OB = 0.694 BU – B OB’ = 0.515

BU – C OC = 0.781 BU – C OC’ = 0.569

Range

for the

better

resource

allocation

The optimal condition for the BU-A is that

the ratio of Labor to Expected Return goes

down to 0.25 and ratio of General Spending

to Expected Return goes down to 0.676;

therefore, we suggest multiple ways for the

BU-A to get closer the DEA model’s efficient

frontier as follows:

- Labor Cost could go down to 3.15%

- General Spending could go down to 3%

- Expected Revenue could go up to 3.07%

Range for

the better

resource

allocation

The efficient frontier generated by the

DEA/P model can be considered to a more

economical best-practice frontier, which

represents a shorter distance from origin to

the efficient frontier.

23 | P a g e

Table 5A – Efficient Score for IT (Project) Portfolio

DEA model DEA/P model

IT Portfolio Efficiency score IT Portfolio Efficiency score Weight score

IT Portfolio – A (BU-A1) E = 0.885 IT Portfolio – A (BU-A1) E = 0.679 0.177 (17.7%)

IT Portfolio – B (BU-A2) E = 0.914 IT Portfolio – B (BU-A2) E = 0.683 0.589 (58.9%)

IT Portfolio – C (BU-A3) E = 0.947 IT Portfolio – C (BU-A3) E = 0.686 0.234 (23.4%)

IT Portfolio – D (BU-B1) E = 0.934 IT Portfolio – D (BU-B1) E = 0.691 0.169 (16.9%)

IT Portfolio – E (BU-B2) E = 1 IT Portfolio – E (BU-B2) E = 0.796 0.188 (18.8%)

IT Portfolio – F (BU-B3) E = 0.932 IT Portfolio – F (BU-B3) E = 0.741 0.643 (64.3%)

IT Portfolio – G (BU-C1) E = 0.889 IT Portfolio – G (BU-C1) E = 0.570 0.167 (16.7%)

IT Portfolio – H (BU-C2) E = 0.960 IT Portfolio – H (BU-C2) E = 0.638 0.123 (12.3%)

IT Portfolio – I (BU-C3) E = 1 IT Portfolio – I (BU-C3) E = 0.776 0.710 (71.0%)

Table 5B – Distance Function for IT (Project) Portfolio

DEA model DEA/P model

IT Portfolio

Distance from

origin to each IT

portfolio

IT Portfolio Distance from origin to

DEA/P’s E-frontier

IT Portfolio – A (BU-A1) OA = 0.752 IT Portfolio – A (BU-A1)

Since both IT portfolio E and IT

portfolio I achieve the optimal

efficiency via the conventional

DEA model, we intend to

further uncover the more

economical resource allocation

by using the DEA/P model.

Thus, OE’ = 0.515 and OI’ =

0.622

IT Portfolio – B (BU-A2) OB = 0.744 IT Portfolio – B (BU-A2)

IT Portfolio – C (BU-A3) OC = 0.736 IT Portfolio – C (BU-A3)

IT Portfolio – D (BU-B1) OD = 0.733 IT Portfolio – D (BU-B1)

IT Portfolio – E (BU-B2) OE = 0.647 IT Portfolio – E (BU-B2)

IT Portfolio – F (BU-B3) OF = 0.698 IT Portfolio – F (BU-B3)

IT Portfolio – G (BU-C1) OG = 0.734 IT Portfolio – G (BU-C1)

IT Portfolio – H (BU-C2) OH = 0.711 IT Portfolio – H (BU-C2)

IT Portfolio – I (BU-C3) OI = 0.802 IT Portfolio – I (BU-C3)

24 | P a g e

Table 6A – Efficiency Score for IT Project

DEA model

IT Project Efficient Score Efficient Score Efficient Score

Aij

A: Org. Dept. A

i: IT Portfolio

j: IT Project

Bij

B: Org. Dept. B

i: IT portfolio

j: IT project

Cij

C: Org. Dept. C

i: IT portfolio

j: IT project

A11 0.697

A12 0.701

A13 0.816

A14 0.692

A15 0.794

A16 0.844

A17 0.688

A18 0.838

A19 0.786

A10 0.855

B11 0.705

B12 0.769

B13 0.711

B14 0.702

B15 0.723

B16 0.687

B17 0.700

B18 0.830

B19 0.795

B10 0.886

C11 0.684

C12 0.693

C13 0.702

C14 0.716

C15 0.696

C16 0.809

C17 0.694

C18 0.724

C19 0.693

C10 0.749

A21 0.732

A22 0.785

A23 0.859

A24 0.656

A25 0.701

A26 1.000

A27 0.839

A28 0.689

A29 0.771

A20 0.693

B21 0.869

B22 0.731

B23 0.710

B24 0.702

B25 0.737

B26 0.693

B27 0.824

B28 0.799

B29 0.782

B20 0.687

C21 0.686

C22 0.711

C23 0.825

C24 0.791

C25 1.000

C26 0.777

C27 0.696

C28 0.863

C29 0.692

C20 0.718

A31 0.731

A32 1.000

A33 0.728

A34 0.747

A35 0.830

A36 0.803

A37 0.771

A38 0.732

A39 0.793

A30 0.800

B31 0.746

B32 0.732

B33 0.699

B34 0.708

B35 0.693

B36 0.713

B37 0.756

B38 0.758

B39 0.726

B30 0.733

C31 0.726

C32 0.712

C33 0.702

C34 0.801

C35 0.801

C36 0.654

C37 0.816

C38 0.708

C39 0.702

C30 0.766

25 | P a g e

Table 6B –Efficiency Score for IT Project

DEA/P model

IT Project Efficient Score Weight Efficient Score Weight Efficient Score Weight

Aij

A: BU- A

i: IT Portfolio

j: IT Project

Bij

B: BU-B

i: IT portfolio

j: IT project

Cij

C: BU-C

i: IT portfolio

j: IT project

A11 E = 0.697

A12 E = 0.701

A13 E = 0.816

A14 E = 0.692

A15 E = 0.794

A16 E = 0.844

A17 E = 0.688

A18 E = 0.838

A19 E = 0.786

A10 E = 0.855

W = 6.70%

W = 2.90%

W = 9.00%

W = 21.4%

W = 2.80%

W = 3.90%

W = 45.5%

W = 14.0%

W = 0.40%

W = 1.50%

B11 E = 0.705

B12 E = 0.769

B13 E = 0.711

B14 E = 0.702

B15 E = 0.723

B16 E = 0.687

B17 E = 0.700

B18 E = 0.628

B19 E = 0.602

B10 E = 0.671

W = 32.8%

W = 2.30%

W = 24.5%

W = 3.00%

W = 1.20%

W = 9.00%

W = 2.50%

W = 6.50%

W = 3.50%

W = 25.5%

C11 E = 0.681

C12 E = 0.533

C13 E = 0.540

C14 E = 0.551

C15 E = 0.536

C16 E = 0.807

C17 E = 0.534

C18 E = 0.721

C19 E = 0.533

C10 E = 0.577

W = 1.50%

W = 4.50%

W = 39.0%

W = 0.10%

W = 9.30%

W = 1.20%

W = 7.30%

W = 9.00%

W = 2.90%

W = 25.2%

A21 E = 0.732

A22 E = 0.785

A23 E = 0.650

A24 E = 0.656

A25 E = 0.701

A26 E = 1.000

A27 E = 0.839

A28 E = 0.689

A29 E = 0.643

A20 E = 0.693

W = 4.20%

W = 2.10%

W = 10.7%

W = 34.3%

W = 17.2%

W = 1.00%

W = 1.90%

W = 20.2%

W = 5.80%

W = 2.60%

B21 E = 0.869

B22 E = 0.731

B23 E = 0.710

B24 E = 0.702

B25 E = 0.737

B26 E = 0.693

B27 E = 0.624

B28 E = 0.605

B29 E = 0.782

B20 E = 0.654

W = 55.9%

W = 20.6%

W = 5.00%

W = 1.60%

W = 2.70%

W = 4.60%

W = 1.00%

W = 2.80%

W = 1.80%

W = 8.50%

C21 E = 0.528

C22 E = 0.547

C23 E = 0.636

C24 E = 0.610

C25 E = 1.000

C26 E = 0.774

C27 E = 0.536

C28 E = 0.861

C29 E = 0.533

C20 E = 0.552

W = 1.80%

W = 25.6%

W = 14.9%

W = 6.10%

W = 8.30%

W = 12.5%

W = 0.3%

W = 2.60%

W = 7.90%

W = 20.0%

A31 E = 0.731

A32 E = 0.749

A33 E = 0.728

A34 E = 0.747

A35 E = 0.629

A36 E = 0.608

A37 E = 0.771

A38 E = 0.732

A39 E = 0.601

A30 E = 0.606

W = 3.20%

W = 2.40%

W = 8.90%

W = 30.5%

W = 1.90%

W = 6.10%

W = 1.80%

W = 13.1%

W = 28.7%

W = 3.40%

B31 E = 0.746

B32 E = 0.732

B33 E = 0.699

B34 E = 0.708

B35 E = 0.693

B36 E = 0.713

B37 E = 0.756

B38 E = 0.758

B39 E = 0.726

B30 E = 0.733

W = 75.5%

W = 2.00%

W = 3.30%

W = 1.20%

W = 3.40%

W = 1.40%

W = 3.60%

W = 2.60%

W = 4.80%

W = 2.20%

C31 E = 0.559

C32 E = 0.548

C33 E = 0.540

C34 E = 0.798

C35 E = 0.798

C36 E = 0.611

C37 E = 0.814

C38 E = 0.545

C39 E = 0.540

C30 E = 0.590

W = 0.4%

W = 2.30%

W = 1.00%

W = 88.2%

W = 0.90%

W = 0.10%

W = 1.60%

W = 2.80%

W = 0.80%

W = 1.90%

26 | P a g e

Figures

Figure 1 – A simulated multi-business unit firm (enterprise) by using the DEA/Parallel model

Frim

(Enterprise)

Business Unit A (BU-A) Business Unit B (BU-B) Business Unit C (BU-C)

27 | P a g e

Figure 2 – Cost Allocation and Expected Return for different organizational levels

(Business Units, IT Portfolios, and IT Projects)

28 | P a g e

Labor cost

Expected Return

Figure 3 – Efficient Frontier for Business Unit through DEA and DEA/P approach

Labor Cost

Expected Return

Figure 4 – Efficient Frontier for IT Portfolio level via DEA and DEA/P approach

General Spending

Expected Return

General Spending

Expected Return

29 | P a g e

Labor

Expected Return

Figure 5 – Efficient Frontier for IT Project level via DEA model and DEA/P model

General Spending

Expected Return

30 | P a g e

Appendix

Table 7 – IT Portfolio Attributes associated with Parameter(s) and Variable(s)

IT Portfolio

Attribute Parameter/ Variable Description Primary Reference Data Source

Benefit

Expected Return The definition of expected return is related to on a

corresponding ROI.

Ilmanen, 2011; Eisfeldt

and Papanikolaou, 2013

Focal Firm IT portfolio data

along with simulated data

Cost Saving

The savings from the business process improvement,

the reduction in inventory, or gathering payables

more quickly after implementing IT project(s).

Kim and Chhajed, 2000;

Lee and Kim, 2000

Focal Firm IT portfolio data

along with simulated data

Budget Cost

Capital Expenditure

The definition of capital expenditure is funds

invested in a firm for the purposes of furthering its

business objectives.

Shim et al., 2012;

Bodmer, 2014

Focal Firm IT portfolio data

along with simulated data

Operating Expense

Operating expense is defined as what a business

incurs as a result of performing its normal business

operations.

Rahman, 1998;

Promislow, 2010

Focal Firm IT portfolio data

along with simulated data

Labor Cost Labor cost (including direct and indirect labor) is

defined as the salaries and wages paid to employees.

Triplett, 2007

Han and Mithas, 2013

Focal Firm IT portfolio data

along with simulated data

Project Type

Must Do

This type of IT project addresses a critical

compliance or controllership issue; these IT projects

receive first priority in terms of funding or resources.

Ross and Beath, 2002;

Crawford et al., 2005;

Kumar et al., 2008

Focal Firm IT portfolio data

along with simulated data Long Term Growth

This type of IT project usually adds new capabilities

for the business; these IT projects have the

significant impact to the existing business process.

Operating Margin

This type of IT project is the ROI-driven IT project

that allows the business to do the same process faster

or at lower cost.

31 | P a g e

Risk Risk

Risk value (risk score) is defined as the probability

that the return falls under the manager’s managerial

expectation for each IT project associated with its

utility function.

Lientz and Larssen, 2006;

Dewan et al., 2007;

Wang et al., 2010

Focal Firm IT portfolio data

along with simulated data

Technical

Complexity

High Technical

Complexity

This category of IT project is to extend applications

to customers or vendors for the first time, introduce a

new business process or new technology that is not

in the standard tech stack.

Tatikonda and Rosenthal,

2000;

Muller and Turner, 2007

Focal Firm IT portfolio data

along with simulated data Med Technical Complexity

This category of IT project is related to major

functionality enhancement, new custom developed

application using non-standard offerings, or new

technology that is not in the standard tech stack but

exists in the current infrastructure.

Low Technical Complexity

This category of IT project is simple functionality

upgrade with standard technologies, new custom

developed application, or significant

capacity/infrastructure expansion.

Portfolio

Distribution

Dominant IT Portfolio

Dominant IT portfolio is defined as a firm that

concentrates its IT investment on one or a very small

number of large IT projects.

Prahalad and Bettis, 1986 Focal Firm IT portfolio data

along with simulated data

Uneven-distribution-based

IT Portfolio

Uneven distribution-based IT portfolio is defined as

a firm that allocates its IT investment to a portfolio

composed of diversified IT projects (e.g., varying

project types and project sizes).

Even distribution-based IT

Portfolio

Even distribution-based IT portfolio is defined as a

firm that allocates its IT investment to all the IT

projects with similar sizes.

32 | P a g e

References

Banker, R. D. and Slaughter, S. A. 1994. “Project Size and Software Maintenance Productivity: Empirical

Evidence on Economies of Scale in Software Maintenance,” in Proceedings of the 12th International

Conference on Information Systems, pp. 279-289.

Banker, R. D., Chang, H., Janakiraman, S. N. and Konstans, C. 2004. “A balanced scorecard analysis of

performance metrics,” European Journal of Operational Research (154:2), pp. 423-436.

Banker, R. D., Chang, H., and Pizzini, M. J. 2004. “The balanced scorecard: Judgmental effects of

performance measures linked to strategy,” The Accounting Review (79:1), pp. 1-23.

Banker, R. D., Chang, H. and Pizzini, M. 2011. “The judgmental effects of strategy maps in balanced

scorecard performance evaluations,” International Journal of Accounting Information Systems (12:4),

pp. 259-279.

Banker, R. D., Hu, N., Pavlou, P. A., and Luftman, J. 2011. “CIO reporting structure, strategic

positioning, and firm performance,” MIS Quarterly (35:2), pp. 487-504.

Bardhan, I., Bagchi, S. and Sougstad, R. 2004. “Prioritizing a Portfolio of Information Technology

Investment Projects,” Journal of Management Information Systems (21:2), pp. 33-60.

Barney, J. 1991. “Firm Resources and Sustained Competitive Advantage,” Journal of Management

(17:1), pp. 99-120.

Barua, A., Lee, S. and Whinston, A. B. 1996. “The Calculus of Reengineering,” Information Systems

Research (7:4), pp. 409-428.

Barua, A., and Mukhopadhyay, T. 2000. “Business value of information technologies: past, present and

future,” Framing the Domains of IT Management: Glimpsing the future through the past, pp. 65-84.

Bharadwaj, A. S., Bharadwaj, S. G. and Konsynski, B. R. 1999. “Information Technology Effects on

Firm Performance as Measured by Tobin's q,” Management Science (45:7), pp. 1008-1024.

Bodmer, E. 2014. Relationship of Growth, Capital Expenditures, Depreciation, and Return on Investment,

in Corporate and Project Finance Modeling: Theory and Practice, John Wiley & Sons.

Brynjolfsson, E., and Hitt, L. M. 2000. “Beyond Computation: Information Technology, Organizational

Transformation and Business Performance,” The Journal of Economic Perspectives (14:4), pp. 23-48.

Charnes, A., Cooper, W. W. and Rhodes, E. 1978. “Measuring the efficiency of decision making units,”

European Journal of Operational Research (2:6), pp. 429–444.

Charnes, A., Cooper, W. W., Golany, B., Halek, R., Klopp, G., Schmitz, E., et al. 1986. “Two phase data

envelopment analysis approaches to policy evaluation and management of army recruiting activities:

Tradeoffs between joint services and army advertising,” Research report Center for Cybernetic

Studies.

Chan, Y. E., Huff, S. L., Barclay, D. W., and Copeland, D. G. 1997. “Business strategic orientation,

information systems strategic orientation, and strategic alignment,” Information systems research

(8:2), pp. 125-150.

Chandler, A. D. and Hikino, T. 1990. Scale and scope: the dynamics of industrial capitalism, Harvard

University Press.

Cho, W. 2010. “IT portfolio selection and IT synergy,” Doctoral dissertation, University of Illinois at

Urbana-Champaign.

Cho, W., and Shaw, M. 2013. “Portfolio Selection Model for Enhancing Information Technology

Synergy,” IEEE Transactions on Engineering Management (60:4), pp. 739-749.

Crawford, L., Hobbs, J. B., & Turner, J. R. 2005. Project categorization systems: Aligning capability with

strategy for better results, Project Management Institute.

33 | P a g e

Dewan, S., and Ren, F. 2007. “Risk and return of information technology initiatives: Evidence from

electronic commerce announcements,” Information Systems Research (18:4), pp. 370-394.

Dia, M. 2009. “A portfolio selection methodology based on data envelopment analysis,” INFOR:

Information Systems and Operational Research (47:1), pp. 71-79.

Eisfeldt, A. L., and Papanikolaou, D. 2013. “Organization capital and the cross‐section of expected

returns,” The Journal of Finance (68:4), pp. 1365-1406.

Färe, R., Primont, D., 1984. “Efficiency measures for multiplant firms,” Operations Research Letters

(3:5), pp. 257-260.

FORTUNE 500 2015. http://fortune.com/fortune500/2015

Han, K., and Mithas, S. 2013. “Information technology outsourcing and non-IT operating costs: An

empirical investigation,” MIS Quarterly (37:1), pp. 315-331.

Hitt, L. M., and Brynjolfsson, E. 1996. “Productivity, business profitability, and consumer surplus: three

different measures of information technology value,” MIS Quarterly (20:2), pp. 121-142.

Ilmanen, A. 2011. Expected returns: an investor's guide to harvesting market rewards, John Wiley &

Sons

Jeffery, M., and Leliveld, I. 2004. “Best Practices in IT Portfolio Management,” MIT Sloan Management

Review (45:3), pp. 41-49.

Kao, C. 2009. “Efficiency measurement for parallel production system,” European Journal of Operational

Research (196:3), pp. 1107–1112.

Kao, C. 2012. “Efficiency decomposition for parallel production systems,” Journal of the Operational

Research Society (63:1), pp. 64–71.

Kao, C. 2014. “Network data envelopment analysis: A review,” European Journal of Operational

Research (239:1), pp. 1-16.

Kaplan, R. S. and Norton, D. P. 2004. “Strategy maps: converting intangible assets into tangible

outcomes,” Harvard Business School Press.

Kaplan, R. S., and Norton, D. P. 2005. The balanced scorecard: measures that drive

performance, Harvard business review (83:7), pp. 172-180.

Keeney, R.L. and Raiffa, H. 1993. Decisions with Multiple Objectives: Preferences and Value Tradeoffs,

Cambridge University Press.

Kim, K., and Chhajed, D. 2000. “Commonality in product design: Cost saving, valuation change and

cannibalization,” European Journal of Operational Research (125:3), pp. 602-621.

Kohli, R., and Grover, V. 2008. “Business Value of IT: An Essay on Expanding Research Directions to

Keep Up with the Times,” Journal of the Association for Information Systems (9:1), pp. 23-39.

Kumar, R., Ajjan, H., and Niu, Y. 2008. “Information technology portfolio management: literature review,

framework, and research issues,” Information Resources Management Journal (21:3), pp. 64-87.

Lee, J. W., and Kim, S. H. 2000. “Using analytic network process and goal programming for

interdependent information system project selection,” Computers & Operations Research (27:4), pp.

367-382.

Lientz, B., and Larssen, L. 2006. Risk management for IT projects. Routledge.

Lucas, Jr., H. 2005. Information Technology: Strategic Decision Making for Managers, John Wiley &

Sons.

Mahoney, J. T. 2004. Economic Foundations of Strategy, Sage Publications, Inc.

Maizlish, B., and Handler, R. 2005. IT Portfolio Management: Unlocking the Business Value of

Technology, John Wiley & Sons.

Markowitz, H. 1952. “Portfolio Selection,” The Journal of Finance (7:1), pp. 77-91.

34 | P a g e

Mata, F. J., Fuerst, W. L., & Barney, J. B. 1995. “Information technology and sustained competitive

advantage: A resource-based analysis,” MIS Quarterly (19:4), pp. 487-505.

McGittigan, J., 2014. “IT Budgeting Key Initiative Overview,” G00262977:2014.

Melville, N., Kraemer, K., and Gurbaxani, V. 2004. “Information Technology and Organizational

Performance: An Integrative Model of it Business Value,” MIS Quarterly (28:2), pp. 283-322.

Milgrom, P. and Roberts, J. 1990. “The Economics of Modern Manufacturing: Technology, Strategy, and

Organization”, American Economic Review (80:3), pp. 511-528.

Milgrom, P., and Roberts, J. 1995. “Complementarities and Fit: Strategy, Structure, and Organizational

Change in Manufacturing”, Journal of Accounting & Economics (19:2), pp. 179-208.

Mithas, S., Tafti, A. R., Bardhan, I. R., and Goh, J. M. 2012. “Information Technology and Firm

Profitability: Mechanisms and Empirical Evidence,” MIS Quarterly (36:1), pp. 205-224.

Müller, R., and Turner, R. 2007. “The influence of project managers on project success criteria and

project success by type of project,” European Management Journal (25:4), pp. 298-309.

Nelson, R. R., and Winter, S. G. 1982. “An Evolutionary Theory of Economic Change,” Harvard

University Press.

Nicholson, W., & Snyder, C. 2011. “Microeconomic theory: basic principles and extensions,” Cengage

Learning.

Penrose, T. 1959. “The Theory of the Growth of the Firm,” Oxford University Press.

Powell, T. C., and Dent-Micallef, A. 1997. “Information technology as competitive advantage: The role

of human, business, and technology resources,” Strategic Management Journal (18:5), pp. 375-405.

Powers, J., & McMullen, P. 2000. “Using data envelopment analysis to select efficient large market cap

securities,” Journal of Business and Management (7:2), pp. 31-42.

Prahalad, C. K., and Bettis, R. A. 1986. “The dominant logic: A new linkage between diversity and

performance,” Strategic management journal (7:6), pp. 485-501.

Project Management Institute (PMI) 2000. A Guide to the Project Management Body of Knowledge

(PMBOK Guide)

Promislow, S. D. 2010. Expenses, in Fundamentals of Actuarial Mathematics, John Wiley & Sons.

Rahman, S. U. 1998. “Theory of constraints: a review of the philosophy and its

applications,” International Journal of Operations & Production Management (18:4), pp. 336-355.

Ray, G., Muhanna, W. A., and Barney, J. B. 2005. “Information technology and the performance of the

customer service process: A resource-based analysis,” MIS Quarterly (29:4), pp. 625-652.

Reyck, B. D., et al. 2005. “The Impact of Project Portfolio Management on Information Technology

Projects,” International Journal of Project Management (23:7), pp. 524–537.

Ross, J. W., Beath, C. M., and Goodhue, D. L. 1996. “Develop long-term competitiveness through IT

assets,” Sloan management review (38:1), pp. 31-42.

Ross, J. W., and Beath, C. M. 2002. Beyond the business case: New approaches to IT investment. MIT

Sloan Management Review, 43(2), pp.51-59.

Shim, J. K., Siegel, J. G., and Shim, A. I. 2012. “Capital Expenditures: Assets to Be Bought, Sold, and

Discarded,” Budgeting Basics and Beyond, pp. 243-266.

Tanriverdi, H., and Ruefli, T. W. 2004. “The role of information technology in risk/return relations of

firms,” Journal of the Association for Information Systems (5:11-12), pp. 421-447.

Tatikonda, M. V., and Rosenthal, S. R. 2000. “Technology novelty, project complexity, and product

development project execution success: a deeper look at task uncertainty in product

innovation,” Engineering Management, IEEE Transactions on (47:1), pp. 74-87.

Triplett, J. E. 2007. “The measurement of labor cost,” University of Chicago Press.

35 | P a g e

Tu, Y. 2014. “IT portfolio characteristics and Choices,” Doctoral dissertation, University of Illinois at

Urbana-Champaign.

Wang, J., Lin, W., and Huang, Y. H. 2010. “A performance-oriented risk management framework for

innovative R&D projects,” Technovation (30:11), pp. 601-611.

Weill, P. and Vitale, M. 2002. “What IT infrastructure capabilities are needed to implement e-business

models?” MIS Quarterly Executive (1:1), pp. 17-34.

Wernerfelt, B. 1984. “A resource-based view of the firm,” Strategic Management Journal (5:2), pp. 171-

180.

Zhu, J. 2003. Quantitative Models for Performance Evaluation and Benchmarking: Data Envelopment

Analysis with Spreadsheets and DEA Excel Solver, Springer Science & Business Media.

Zhu, K., and Kraemer, K. L. 2002. “E-commerce metrics for net-enhanced organizations: Assessing the

value of e-commerce to firm performance in the manufacturing sector,” Information Systems

Research (13:3), pp. 275-295.

Zhu, K. 2004. “The complementarity of information technology infrastructure and e-commerce

capability: A resource-based assessment of their business value,” Journal of Management

Information Systems (21:1), pp. 167-202.