Relations Among Measures Climate of Control and PM Models

Relations among Measures, Climate of Control,and Performance Measurement Models*

MARY A. MALINA, University of Colorado at Denver

HANNE S. O. N0RREKLIT, Arhus School of Business

FRANK H. SELTO, University of Colorado at Boulder

1. Introduction

This study reports the evolution of an investigation of the cause-and-effect proper-ties of a performance measurement model (PMM) developed by a Fortune 500company for its North American distribution channel. The company and this studyrefer to the PMM as the distributor balanced scorecard or DBSC. The study followsprevious, related research and also is motivated by balanced scorecard (BSC) liter-ature that stresses the importance of the cause-and-effect properties of balancedscorecards (e.g., Kaplan and Norton 1996, 2001; Ittner and Larcker 2003) andempirical research that suggests causality (Bryant, Jones, and Widener 2004; Ittner,Larcker, and Meyer 2003; Banker, Potter, and Srinivasan 2000; Ittner and Larcker1998; Rucci, Kim, and Quinn 1998). Previous research by Mallna and Selto 2(X)1,2004 has established that the company implemented the DBSC to communicateand match its new customer-service strategy; to provide a more diverse, accurate,and balanced set of performance measures; and to direct distributors' decisionmaking. Extant research implies that a well-specified PMM reflects a firm's pro-duction function and that cause-and-effect relations among measures drive controleffectiveness. We review how cause-and-effect relations among performance mea-sures are beneficial for control purposes. The present study then proceeds as aneconometric validation of the cause-and-effect properties of relations among mea-sures of the DBSC. However, refutation of cause-and-effect in the DBSC leads toconsideration of alternative explanations for the company's continued use andprofessed satisfaction with the DBSC. These plausible, internally consistent alter-natives provide motivation for future research that might support cause-and-effectproperties in other PMM, the alternative explanations, or both.

Accepted by Sieve Salterio. The authors appreciate comments and suggestions from workshopparticipants at Rice University. Universily of Vermont. San Diego State University. Arhus Schootof Business. tJniversily of Titburg, University of Colorado at Boutder. University of Ghent, iheManagement Accouniing Research and Case Conference 2(X)5, and the North American FieldResearch Conference at Queen's University 2005. We thank Qiuhong Zhao. Yanhua Yang, ^ldVeronda Willis for research assistance. We gratefulty aclmowtedge significant contrituitions froman associate editor and two anonymous reviewers.

Contemporary Accounting Research Vol. 24 No. 3 (Fall 2007) pp. 935-82 © CAAA

doi:10.1506/car.24.3.10

936 Contemporary Accounting Research

Research questions

This study begins with the following research question:

RESEARCH QUESTION 1, Do DBSC relations from the distribution strategymap exhibit valid cause-atid-effect properties?

We analyze the company's distribution strategy by investigating company doc-uments and transcripts from interviews with five distribution managers and DBSCdesigners and nine distributors. We initially look to these data for evidence of cuu-sality in its DBSC. From the qualitative data, we document perceived linkagesamong the DBSC's performance measures that were validated by managers. Theelicited strategy map generates testable, cause-and-effect relations, which aredescribed in detail later. We test these relations using 31 quarters of performancedata (1997-2005) and multiple tests for cause and effect. Overall, few hypothe-sized leading-performance measures in the DBSC explain lagging measures, andnone of the estimated model relations containing hypothesized performance drivershas significantly better predictive ability compared with models containing onlylagged dependent variables (that is, causality tests by Granger 1969. 1980). Yetdespite the refutation of causality by empirical tests, the company and its distribu-tors expres.s .satisfaction with the DBSC and plan to deploy it worldwide. Hence,we continue the study with a second research question:

RESEARCH QUESTION 2. Are statistically significant cause-and-effect relationsnecessary for effective management control?

We consider alternative explanations for the apparent ongoing success of theDBSC through the lens of management control theory. To do so. we expand ourqualitative data through additional analyses, interviews, and review of companydocuments (that is, data not in Malina and Selto 2001). Importantly, additionalqualitative analyses revise our prior conclusion (Malina and Selto 2001), and wefind that the relations among performance measures perceived by DBSC users arenot cause-and-effect relations. In addition to the previously supported communica-tion benefits, the reanalyzed qualitative data provide evidence that managers anddistributors regard the DBSC as an effective management control because itscommunicated relations among measures create a complementary (a) crediblestory of success, (b) reinforcement of the company's pay-for-performance culture,and (c) result control that is legitimate and fair. The company has used the results ofthe DBSC to guide consolidation of distributorships from 31 to 19, and continuingmanagers appear to alter strategic and operational choices consistent with theDBSC measures, both without statistical evidence of reliable cause-and-effect rela-tions among the measures.

We conclude that managers' beliefs about relations support the organization'sclimate of control and drive the design and continued use of the DBSC. We alsotentatively conclude that statistically valid cause-and-effect relations may beurmecessary to achieve desired control effectiveness in this context and perhaps in

CAR Vol. 24 No. 3 (Fail 2007)

Measures, Climate of Control, and Performance Measurement Models 937

otbers. Although tbis result seems surprising in light of tbe normative PMM litera-ture, tbe expectation of cause-and-effect relations may reflect common assumptionsrather than evidence. Organizations may use dynamic PMMs tbat are composed ofrelations that are not cause and effect, but may be more than comtnon sense, tofacilitate strategic communication and to create a climate of control rather thanto create a predictive business model for use as a decision aid. business simulation,or input-output model (e.g.. Zimmerman 1997, 4-5). Perhaps a predictive busi-ness model is the least important reason for a PMM.

Tbis study next reviews relevant cause-and-effect relations literature. In section 3,tbe study tben reports the qualitative modeling of cause and effect in the DBSC and.next, in section 4 econometric efforts to refute cau.se and effect, wbich were success-ful. The study, in section 5. prtx:eeds with the evolution of tbe inquiry by developingplausible altemative explanations and shows that the econometric results and alterna-tive explanations challenge common assumptions about the existence and import-ance of cause-and-efiect relations in PMM. In section 6. we present our theoreticalframework of PMM control effectiveness. This mtxiel can serve as a point ot depar-ture for future research, as described in tbe final section of tbe study.

2. Importance of cause and efTect in PMM

Proponents of PMMs invariably cite their inherent cause-and-effect relations as amajor source of the value of sucb models. We wish to precisely define what wemean by cause and effect because it is not clear tbat all PMM researchers use acommon definition. Most scientists and theories of science adopt Hume's criteriafor a cause-and-eftect relation (Cook and Campbell 1979; Edwards 1972. 2:63: Slifeand Williams 1995; N0rreklit 2000). and this study also adopts tbem. The criteria,wbicb are restrictive, are (a) independence, (b) time precedence, and (c) predictiveability. Tbe independence criterion states that events X (the cause) and Y (the effect)are logically independent. Furthermore, one cannot logically infer Y from X butonly can do so empirically. Tbe time-precedence criterion states that X precedes Yin time, and tbe two events can be observed close to each other in time and space.Tbe predictive-ability criterion is tbat observation of an event X necessarily impliesthe subsequent observation of the other event Y.

Cause-and-effect relationships are well known in physical sciences and likelyexist in firms' physical production functions. For example, a cause-and-effect rela-tionship exists between applied beat and tbe temperature of water. The heat of afire and the temperature of water are independent phenomena, and a rise in watertemperature occurs after the application of heat. Furthermore, one can predict thewater's future temperature from the observed rate of beat transfer using a theor-etically based cause-and-effect relationship. Similarly, firms in many industriesmay develop PMMs tbat (partly) reflect underlying physical processes.

Benefits of cause and effect in PMM

For several decades tbe strategic management literature has presumed the exist-ence of cause-and-effect relations among key performance indicators (KPIs) ormeasures at various levels of the firm.' Although physical processes such as those

CAR Vol. 24 No. 3 (Fall 2007)


in chemical industry are analogous to heating water, many KPI relations can bemore complex and less deterministic. Nonetheless, the notion of cause and effectamong KPI is widespread. For example, Porter (1985) revolutionized .strategicmanagement with the application of the value-chain concept, which links KPIalong the product and service delivery chain. Kaplan and Norton (1992) introducedthe notion of a causal balanced scorecard, which has influenced the managementaccounting literature and which is a direct descendant of the value-chain and sys-tems models.2 These seminal works argue that cause-and-effect relations exi.stamong proper KPIs, and all of the supporting literature identifies process andoutcome benefits from building PMMs with cause-and-effect relations. We brieflydiscuss these benefits, which include predictive ability, improved decision making,communication, leaming, and goal congruence.

Predictive ability

Cause-and-effect relations, by their nature, use leading indicators to predict keyoutcomes. If reliable, predictive relations exist in PMM, for example, leading meas-urements in nonfinancial areas can be used to predict future financial pertbnnance(Kaplan and Norton 1996, 8). Furthermore, analytical models demonstrate that theevaluation weighting of measures can depend on their predictive ability (:uid deci-sion sensitivity: e.g., Datar, Culp, and Lambert 2(K)I). Goal setting and expectancytheory research (Locke and Latham 1990; Green 1992) demonstrate that individu-als are motivated to earn incentives when they believe that their efforts drive per-formance measures (and also when goals are achievable and rewards are based onmeasured performance). Multiperformance measure systems can be useful man-agement controls, but they are not easily interpreted unless one can describe how achange in one criterion affects a change in another (Ridgway 1956). Thus, if rela-tions in PMM meet Hume's predictive-ability criterion for cause and effect, theyclearly can be useful to develop and control reliable planning scenarios.

Improved decision making

Related benefits also can accrue inside the "black box" of predictive ability. Reli-ably predicting future effects of current actions and outcomes at key points in thevalue cbain can aid decision making (e.g., Eccles 1991). Resource and capability-based strategy research predicts tbat superior decisions and performance will resultfrom systemic management, rather than myopic focus on individual elements ofthe value chain (e.g.. Huff and Jenkins 2002; Sanchez, Heene, and Thomas 1996;Forrester 1994). A PMM with valid, predictive relations is posited to reduce thecognitive complexity of both understanding and managing multiple measures ofperformance (Luft and Shields 2002; Morecroft, Sanchez, and Heene 2002). Further-more, a predictive PMM can free managers to focus more on strategic and evalua-tion decisions than on information processing (e.g., Kaplan and Norton 2001).

Communication

Cause-and-effect relations can enable effective communication of how best toachieve key operating and strategic performance. From a systems perspective,



de Geus (1994) argues that even a simplified but credible PMM can be a powerfulcommunication device. Magretta (2002) also argues that models to explain anorganization's business activities are essential to tying strategic choices to fmancialresults (see also Ittner and Larcker 2001). Morecroft and Sterman (1994) furtherargue that PMMs are effective when they become integral parts of managementdebate, dialogue, communication, and experimentation. Indeed, facilitating andcommunicating strategy by means of demonstrated cause and effect are some ofthe key "'selling points" of Kaplan and Norton's 1996, 2001 balanced scorecard.

Learning

The cause-and-effect relations in a PMM demonstrate outcomes and trade-offsamong leading and lagging measures. Nonaka (1994) and Nonaka and Takeuchi(1995) argue that successful organizations institutionalize and perpetuate learningthrough creating, capturing, and communicating critical knowledge. PMMs withcause-and-effect relations can educate managers and help them in controlling andcommitting to muhiple measures (e.g., Feltham and Xie 1994; Willard 2005, 131).

Goal congruence

Incentives based on single measures can induce incongruent behavior and manage-ment myopia (e.g., Ridgway 1956; Dearden 1969). Because a cause-and-effectPMM helps individuals to see how their actions affect future performance, it fostersorganizational focus and goal congruence (Kaplan and Norton 2001, 2005). Astrategy-driven PMM guides individuals to formulate IcKal actions that conlributeto achieving organizational-level strategic objectives. Hence, cause-and-effect rela-tions direct managers' decisions to align tbe organization's limited resources withstrategic outcomes.

Summary ofempirieal evidence for expected benefits

The beneficial effects of cause-and-effect relations allegedly support improvedpredictions, decision making, communication, learning, and goal congruence.These outcomes should be observable in PMM users' strategic and operationalcboices and in operational and financial outcomes. Although influential literatureclearly points to cause-and-effect relations as essential for the success of PMM,empirical support is minimal.

The few empirical studies of the existence or benefits of cause-and-effect rela-tions in PMM are inconsistent. Contrary to Malina and Selto 2001, botb Bankeret al. (2000) and Ittner and Larcker (1998) find that relatively few managers andexecutives in their sampled firms had learned or understood any cause-and-effectrelation between customer satisfaction and future profitability, altbougb Iheirincentive plans were linked to both. Lipe and Salterio (2002) find that experimentalsubjects made different but not necessarily better decisions related to alternativeformats of performance measures (that is. randomly arranged measures versusmeasures in displayed "balanced scorecard" categories). Ittner and Larcker (2003)observe thai cause-and-effect relations among firms' multiple performance meas-ures often are neither specified nor measured well. They find that companies rarely

CAR Vol. 24 No. 3 (Fail 2(K)7)


associate the actual impacts of changes in nonfinancial measures with future financialresults. Bryant et al. (2004) associate cross-sectional data that proxy for outcomemeasures across four typical BSC perspectives to explain financial performance. Ina more powerful test. Banker et al. (20(M)) use context-specific, time-serie.s data toprovide evidence on the impact of nonfinancial measures on firm performance.Neither of the latter studies tests for cause and effect, but both document sugges-tive associations between customer satisfaction and future financial performance.Empirical evidence that supports the predictive ability of PMM has been in theform of uncritical self-reports (e.g., Rucci et al. 1998). Indeed, most systemsexperts downplay the long-term predictive ability of complex systems models(e.g.. de Geus 1994). Hence, exploring evidence for the existence and benefits ofcause-and-effect relations is the original motivation for this study.

3, Research site and cause-and-efTect model development

The host company for this study, a Fortune 500 firm, has sponsored two previousstudies (Malina and Selto 2001, 2004).^ These earlier studies relied almost exclu-sively on qualitative analyses of extensive interviews with company and distributionmanagers. The findings of the two previous studies motivated the present study. Inbrief, the previous studies document that managers and distributors perceived thatthe DBSC:

1. contains credible cause-and-effect relations among DBSC measures, althoughthe company has neither expressed the relations as a "strategy map" nor con-ducted statistical testing of the relations;

2. communicates strategic intent effectively;

3. promotes goal congruence by effective communication and incentives toachieve strategic objectives;

4. directs distribution managers to change their processes and decisions toachieve DBSC targets;

5. failed to achieve the above when communication was ineffective; and

6. has been revised repeatedly as the company seeks to include only accurate,reliable, and auditable DBSC measures.

More recent interviews have disclosed that the company plans to deploy theDBSC to its global distribution network. The accumulated evidence leads the authorsto believe that the DBSC is an example of an effective PMM that possesses qualitiesdescribed in the normative BSC literature. The qualitative support for cause-and-effect relations in the DBSC (finding I above) is particularly motivating for thisstudy. Thus, this study was initially motivated to answer what we believed was theonly unanswered research question: Whether statistically reliable cause-and-effectrelations actually exist in the DBSC. But we uncovered other interesting questionsin the process.


Measures, Climate of Control, and Performance Meastirement Models 941

The DBSC data set

When the DBSC was introduced in the fourth quarter of 1997, it contained morethiui 20 key performance measures. After two years of evolution, the DBSC droppedto 11 somewhat different performance measures. A time-line of DBSC events isshown in Figure 1. Across the 31 quarters of data comprising this study (1997-2005),seven measures have been used continuously and have sufficient data for the statis-tical analyses that appear later in this study. Since Malina and Selto 2001, thecompany has reduced the number of distributors to 19 by merging lower-performingunits with higher performers. The 19 surviving distributors have up to 31 consecu-tive quarters of data. All available performance data are used in the analyses thatfollow. Table I contains the continuously used DBSC measures and brief defini-tions and explanations of the sources of the measures.

DBSC model development

The company had not expressed its DBSC as a strategy map, which is a prominentfeature of the balanced scorecard literature. We derived the DBSC map for thisstudy from interview data using a method identical to the first method reported byAbemethy, Home, Lillis, Malina, and Selto 2005. The method analyzes the elicitedknowledge of individuals within an organization first by coding interview tran-scripts for revealed performance constructs. We initially used Malina and Selto's2(X)1 coding of semi-structured interviews with five DBSC managers and nine dis-tributors to determine the relations between pairs of measures in the DBSC thatusers and managers perceive. In total, 179 coded comments referred to variablerelations.'* Following the PMM literature and our earlier work, we inferred cause-and-effect from interviewees' comments, and we initially coded 84 of thesecomments as cause-and-effect relations between specific pairs of variables.^ Thesummary of computer coding in row one of Table 2 generates the constructs orbuilding blocks of the hypothesized cause-and-effect model.

The second step of the qualitative method to build a cause-and-effect map is toobserve consistent patterns or relations among the coded constructs using rela-tional queries in qualitative data base software.^ Related constructs are connectedwith directional arrows, which we inferred from the nature of the relation comments.In addition, we subjectively evaluated each relation for consistent expressions ofrelations rather than merely unrelated proximity. We validated this model by pre-senting it to two company managers, who were responsible for the administrationof the DBSC and approved the model. Hence, we believe that we have properlyspecified the company's beliefs for cause-and-effect relations in the DBSC.Figure 2 is the visual representation of the DBSC.

Figure 2 describes the cause-and-effect performance model that company per-sonnel perceive as a map of organizational success. The time periods and extendedboxes of Figure 2 reflect approximate temporal, lagged effects, which are positedto be integral to a PMM's cause-and-effect validity (N0rrekUt 2000). Managersand distributors expected time lags in the identified relations but could not be pre-cise about the length of the lags. Distributors expect lags of one to two quarters.

C/U? Vol. 24 No. 3 (Fall 2007)


H

_- Q

rQ

C/U? Vol. 24 No. 3 (Fall 2007)

Measures, Climate of Control, and Perfonnance Measurement Models 943

— O

4J 3

^ eE o

-ISt u

red

•3 ^

153 ii>.

002u b

sn <J

I I

E ESi E

C Bo o•a -3

IM

.5

" 2P

S I

i I.E S3

a> „

-o .S

t -S

a, a. ^ D.E e i E0 0 • - oU U Q U

a.oO

S" o 3U XI

c i s_> ^ Cq "JT i+..i

a. (N cu u o

3 U

•s "i

o S)S tf]

S2 1 3

iI 2 -2 2*" (0 " O

MilE -o--• M t-"9 ^ RJ U w

i> S•o =

.b

&f

10z

avai

ial

the

thi

0

1 aly

sis

0

y

E0

x :

y

en

"•s

i£•a

nu

/asc

?

i• '

grow

mU

13

êra

.

ra-0u.H00

iabl

e w

ei

1u•ScS

u0Z

S.A

lls

uc

100di

1u

u l

c

an

gbi«

0

i g

0

com

u

ingi

mca

T3

TJ

t yi

el

•5

•S0

•g

•0

£

^par

ts, s

eur

es (

E

ipor

ted

u•Ji

0•S

•S

esul

ts

u

iff

-a

ical

lto

to0

c

10

ithn

S3J

00

•s

grow

les

av>-0U

1i n

ited

wit

hre

pe:

u

buto

rs.

1Ip SSOJ

renc

es

_ j

u

•odu

ex

1i£a

ifisi

gn

0

I f ]

cat

t s

0bb

130 )

'Sa.

LA

in re

sult:

renc

e

. 24 No. 3 (Fall 2007)


UCOQ

an

I

2

ii

• Is t j

1^S 3

CU U

o —i —.

O —I

O — C

— O in O

C —• O

I

a u

.3

> oCO X

x: o SJ00 en S.

^ ' i ^« 3 5 ^ gCU U c£ ? e-

"S• -

.5

.co•q

"6

O

2-a

cT3coa.u*-•x:

EUV I

Qu

1coo

sed

3

a;u

• o

3

hE

f 1—1

. ^

a.1/1

o

EN

IuE

o

cu• oc0I

2-o

"- O< 1=

1^1I

•So

oa

1o.c

CAft Vol. 24 No. 3 (Fall 2007)


perhaps up to one year, for the effects of early value-chain performance measures(for example, fill rate to customer satisfaction, customer satisfaction to sales growth).

4. Testsof cause and effect

Despite widespread beliefs in cause-and-effect relations in PMM. statistical valida-tion of causality is not trivial. Empirically verifying cause and effect requireseffective experimental controls that rule out alternative explanations and permitcause-and-effect inferences. Clearly, one cannot infer causality on the basis ofcovariation between variables. Although simultaneous cause and effect mightexist, without careful controls one could not rule out that an unobserved variablewas the cause of simultaneously observed effects. Time-series models of effectsalone cannot provide evidence of causality; they test only for temporal precedence.Finally, predictive-ability demonstrations are insufficient to support causality; theydocument out-of-sample regularities. A systematic, holistic approach is indicated.Hence, we employ a weII-validated, rigorous econometric approach. Granger caus-ality, to detect cause-and-effect relations in the DBSC.

Figure 2 Management's and distributors' expected DBSC relations*

Time 1

Fill rate {FR)

\

Time 2

Customersatisfaction

(CSAT)*

Tune 3

Weighted averagesales growth

(WASG)

Safety {SAFE)

Parts inventory turnover {PTO)

Whole goods inventory turnover {WTO)

*

/

Time 4

Profit beforeinterest and tax

(PBIT/S)

fNotes:

* The distributor's customer fill rate affects parts inventory tumover Order till rate isexpected to affect customer satisfaction, because parts availability affects howquickly the distributor can meet customers' parts and service needs. Note that thecompany's mea.sure of customer satisfaction is obtained during the quarter ii isreported, and it might have more Immediate impact than is observable, just byconstruction. The company believes the best means to drive sales growth (anddistributor profitability) is through improved customer satisfaction. Safety affectsprofitability through insurance costs and lost billable time. Safety and the tumoverof inventories have direct impacts on distributor profitability. Time period.s arerelative and are not intended to accurately reflect quarterly effects.

Source: Coded interview transcripts from Malina and Selto 2001.



Granger causality

The fully developed concept of "Granger causality" (Granger 1969, 1980; Ashley,Granger, and Schmalansee 1980) is consistent with Hume's criteria and dominatestesting for cause-and-effect evidence in economic models. The method proceeds intwo steps. First, Granger causality is inferred from X loY when significant correl-ation is observed between X and Y while considering all available sources ofinformation. This condition supports or refutes the uniqueness of the relation oralternative explanations. Operationalizing such tests literally is impossible inarchival, quasi experiments because "all available sources of information" cannotbe controlled or measured. However, tests of Granger causality customarily regressa dependent variable on lagged values of the dependent variable, Y, assuming thatlagged values of K and the hypothesized lagged independent variables capture "ailavailable information". Granger estimation tests support causality if coefficients oflagged independent variables, which capture time precedence, are significant aspredicted in the presence of the lagged dependent variables (Damell 1994). Second,Ashley et al. (1980) propose that more rigorous Granger mean causality is inferredif the mean squared error of a forecast of K is significantly less using a model oflagged X and Y (the full model) than using only lagged values of K (the constrainedmodel). If the full models have superior predictive ability, their root-mean-squaredprediction errors (RMSEs) and residua! sums of squares (RSSs) should be signifi-cantly smaller than those of the constrained models. Granger causality can measuretheory, temporal ordering, high correlation, and predictive ability, which are thenecessary elements of causality. The Granger tests we implement here (as in mostarchival studies) might support reliability, but can only refute cause-and-effectvalidity. This is consistent with most conventional notions of scientific inquiry thatseek rejection of null hypotheses.

Hypothesized cause-and-effect DBSC relations

The optimal lag structure of the DBSC is noi apparent theoretically or from theinterview data, but time-series models (not tabulated) indicate a consistently sig-nificant (a = 0.05) one- and two-quarter lag structure in the DBSC's dependentvariables. We conservatively include dependent and independent variables laggedup to four quarters in the following tests to capture time precedence and "all avail-able information". The relations of the DBSC can be expressed as a system of linearpath equations, which are derived from Figure 2:

PTO, = flo + XhiPTOj + icjFRj + €, (1),

CSAT, = do+ XeiCSATi + XfjFRj + y, (2),

= gQ + XhiWASGj + ikfSATj + S, (3),

= /() + XmjPBIT/Si + injWASGj -\- XojPTOj + '^j + ^, (4),



where PTO is parts inventory tumover, FR is customer parts fill rate. CSAT is cus-tomer satisfaction, WASG is weighted average sales growth, PBIT/S is distributorprofit before ititerest and taxes divided by sales, WTO is whole goods inventorytumover, SAFE is safety, and e,, y,, S,, and ^, are independent, normally distributederror terms.^ Right-hand-side summations (S) of the lagged dependent variables arefrom 1 = / - 1 to r — 4, and summations of the independent variables are fromj = t io t — 4. Granger causality tests require that the lagged independent variablesare significant and, in this case, that all but one of the variable coefficients havepositive signs, because of the nature of the posited relationships. The exceptionsare coefficients qj on SAFEj, which are expected to be negative.

Quantitative data

We originally had 14 quarters of data available to estimate the DBSC's relationsand quarters 15-17 to use as a holdout sample to test the DBSC's predictive ability.The initial tests of multiple, altemative specifications were unsupportive of causal-ity in the DBSC (see Table 3), with only one statistically significant, hypothesizedcause-and-effect relation, which indicates that sales growth, lagged four quarters,might cause distributor profitability in a linear Granger model. However, all of thetested relations have uniformly inferior predictive ability (not tabulated), refutingGranger causality. This evidence points to a noisy model that a successful firmclings to for no apparent good reason.

Since the time of Ihe initial analysis, the company has refined both measuresand measurement methods to improve the accuracy and verifiability of DBSC per-formance (Malina and Sello 2004). Therefore, we have reason to believe thatanalysis of an expanded and improved data set that is now available might revealthe expected cause-and-effect relations among DBSC measures. The expandeddata set includes 31 quarters of DBSC data (1997-2005), which include the 17quarters used initially. Analogous to the initial study, we use 28 quarters of data(QI-Q28) to estimate the DBSC relations and the remaining three (Q29-Q31) asa prediction sample. Since the initial analysis, 9 of the 31 distributors were mergedwith larger and better-pertonning distributors by the third quarter of 2004 (quarter28); two others were merged one quarter later; and one was merged two quartersafter that, leaving 19 distributorships. All available data are used to estimate eachof the DBSC relations because the expected relations should apply to al! distribu-tors, regardless of perfomiance or merger status.^

Descriptive statistics and pair-wise correlations for the estimation set of theexpanded (unlagged) data are presented in Tables 4 and 5, respectively.^ Exploratoryfactor analysis of the seven (unlagged) DBSC variables simultaneously indicatesthat further data reduction is not necessary (results not tabulated). Correlations inTable 5 are generally small and indicate lack of multicollinearity.

Granger estimation tests using the full data set

Column two of panels A, B, C, and D of Table 6 presents the linear Granger testresults of DBSC relations for the full data set. These results show improvementover the initial data set, with some statistically significant relations among DBSC

CAR Vbl. 24 No. 3 (Fall 2007)


ono

UonCQQ

CQ g

o

a- "

I

CQ

CQ

CQ

CQ

§d

•s

1Iou

qd

od

m «n 00 m(N -O r "nvO O 00 >O

S - * C3v (M•* Tf 00

d dor-~

Ol o r4 m m

q ^ * " "d d d d d

O OS g —

— — ^ r -

o

d

OC

dr-d1

O O

— d1

O l<*!

d

o

eff

fixed

.882

o^ 00 CTi

— oo p

0.423

164

698 01

927

0.654

078

-1.099

121

0.597

045

1.325

665

-0.067

032

o o o o o o o o

^ N n ^

g S g g gU O. Q, a. Q,

CA/? Vol. 24 No. 3 (Fall 2007)


LU BQ

o

ffl

oq

CQ

en

pd

" cJ d — — d

00 O m r j — O O00 r-- 1/-. ^ o^ — m•* — p q — — pd d d d d d d

I I ICO

m O o p —K-) C — — Oo S d zi c5

I I I

p pd d

— d r - r j r - — « n Oi - ^ O C v O O f l Q V l Mi n o \ D ( N r j p f ^ » n— >n — d d r j — d

— o

d d

•auX

iZ

0=

led

•5<

. 24 No. 3 (Fall 2007)

950 Contemprorary Accounting Research

o

CiO

O

CQ

CQ

CQ

g.

o1

00

d1

m O1

^ :s— d

t

1

Tt ^

d d1

—I

O^

_JI

o

r-

,_!

O1

d1

CNOO

^

Omd

X

C^ " ^ ^ ^ ^5 ^ ^ ^ j (^ -< ( i t

I I I

Q r-— d d d d

r** o o "^ o ^ " ^ m OS> n > n o > n — 3 ' 3 — Ô ^ C J O O O P — O

o o o o o o o o

u


Meastires, Climate of Control, and Performance Measurement Models 951

oU

uJ

03

00

II

O

I

^ o ^Ô V"^ r o

d — —

s s sd a d

o 'C

o o o

O r l O O r - l r M O O O — o o — ( N O O O O OI I I I I i I

O (N oOo — fOp p p

8 Sd d

o o — —

r f — O ' N ' O r S m ^ O — — — —r* — T*.- f^ ^ j ^^ ^p 1/3 ^^ ^^ ^^ ^5.—* o o o o o o o S o o odddd-cicid^ddcid

I I 1 I I I I

o o o oI I

-J fc

8-^

» N 0 3 ; v ^ o o o o u | 3 ^ D o o —^r ^^ ^ i v^ D 00 f^ c i ^ i r^r - i o o r ^ i O r ~ - ^ " N a c ^ O

o

537

o1

410

o1

r-

7

o1

382

T

I I

0=

O CD CQ CQ CQ Û Q. Q. a , O. ^

< < ^ ^

i i i iP ft

to to i i i

. 24 No. 3 (Fall 2007)


TABLE 3 (Continued)

Notes:

Models and their descriptions:

Granger Granger estimation models reported in the paper, up to 14 observationsper distributor.

Granger w/fixed Granger models with 30 fixed distributor effects (not shown), up to 14observations per distributor;

Log-linear Log transformed models, no fixed effects, up to 14 observations perdistributor;

1 -quarter change Qianges model, first differences, up to 13 observations per distributor,

and

4'quarter change Changes model, fourth differences, up to 10 observations per distributor.

Only significant, lagged observations of performance drivers might be interpreted ascausally related to performance (shaded rows). Because first- or fourth-differencevariables also contain the contemporaneous value, their coefficients are ambiguousabout causality.

* Variables in shaded rows are hypothesized causes of performance. Coefficients in

bold are significant and signed as predicted C < 005, * < 0.01, S < 0.001).

* FR. FRl, and FR3 are highly collinear {R > 0.70).

CSATand CSATl are highly collinear (R > 0.70).

tt PT0J-PT04 and WT0J-WT04 are highly collinear (R > 0.70).

TABLE 4Performance measure descriptive statistics: Full data set (Q1-Q31)

Performance measures

FRPTOWTOSAFECSATWASCPBIT

Complete n (listwise)

/;

760856856736784856855

700

Mean

0.8204.4628.9983.2020.7660.2350.048

Min.

0.0101.1001.3000.0000.000

-0.533-0.016

Max.

0.97825.30036.40023.100

1.00027.3900.206

s.d.

0.0751.5754.6472.8150,0961.0530.023

Notes:

* Includes five outlying observations of WASG.

Variables are as defined in Table 1.

CAR Vol. 24 No. 3 {Fall 2007)

Measures, Climate of Control, and Pertormance Measurement Models 953

mea.sures in the predicted directions. Parts fill rate (FR) in panel A, column 2, doesnot, however, cause parts turnover {PTO). In panel B, till rate (FR) has a statistic-ally significant, contemporaneous association with customer satisfaction (CSAT),as believed by company personnel (p < 0.01), hut no lagged effects that supportcause and effect. In panel C, customer satisfaction (CSAT) does not cause salesgrowth {WASG). In panel D, contemporaneous sales growth {WASG) and partsturnover {PTO) are associated with distributor profitability (PBIT/S), as believed{p < 0.01), but these associate current variable values and do not support causality.However, the four-quarter lag of sales growth iWASG4) appears to cause distribu-tor profitability (p < 0.001).'" Therefore, the Granger estimation tests indicate apossible cause-and-effect link in the DBSC: distributor profitability, PBIT/S,might be caused by one-year lagged sales growth, WASG4.^'

Granger predictive ability tests using the full data set

If the full models have superior predictive ability, their RMSEs and RSSs shouldbe significantly smaller than those of the constrained models; that is. all percentagedifferences in Table 7 should be significantly negative and all F-statisiics shouldexceed critical values. The predictive-ability results were prepared as follows:

1. Estimate each of the four out-of-sample outcomes {PTO,, CSAT,, WASG,, andPBITIS,) using the full, estimated equations in Table 6 (including all hypoth-esized lagged variables).

2. Estimate each of the four out-of-sample outcomes using constrained equationsthat contain only the lagged dependent variables (constrained equations notshown), which provide predictive-ability benchmarks.

3. Compute and compare RMSEs and RSSs across the pairs of equations foreach dependent variable observation.

TABLE 5Piiirwise Pearson correlations of unlagged variables: Full data set (Q1-Q31)

tQl-Q31;

FRPTOWTOSAFECSATWASG

707 s /I <

PTO

0.093*

856)

WTO

-0.072'0.363+

SAFE

0.094'0.046

-0.030

CSAT

0.088*0.150+0.012

-0.008

WASG

-0 .081 '-0.019

0.0010.163+0.040

PBITIS

0.107+0.221 +0.178+0.U5+0.038

-0.015

Notes:

* Correlation is significant at a = 0.05 (two-tailed).

+ Correlation is significant at a = 0.01 (two-tailed).

Variables are as defined in Table 1.



o•B

uCQQ

i

3

•4

o

a

o:

CO

I

r-00

d

o1

ocd1od1d

—rj1

vc o r

I I

+1- ++— r'-. o r^ r- o „o — — "

— I

>r; r— O — fOrJ rN d — •—

Q 00 0000 00 (*-,

p CN — •^. O; oo wd d d d d d d d ~ ~

I II I

d

d

—

—'.

IO

d1

ood

• — *

8d

OrN

1

rN

rid

rJIO

1

fN

d

o

00

d

f*_^

1

00

d

d

I I

• <

"3e0.

1coU

po.

P S =*: CL k, '*•

ins

T3<*


Measures, Climate of Control, and Petformance Measurement Models 955

oU

CQ

00

I

o

CQ

ffl

CQ

o00

pd

.889

o1

rN

od

819

o

,629

CN

>od

CN

,348

grJd

cnDOen

,316

as

od

462

o

,132

d

705

o

,950

fN

—

d

CN

mfN

,588

T___-d1

257

_

,114

o1

odt

255

o

p i o — p p — i p pd d d d d d d d

218

CN

109

o

952

10.

508

o

194

cn

161

o

752

o

038

o

534

069

o

176

m

203

o

718

7

,112

o1

063

-0.

003

-0.

II. 24 No. 3 (Fall 2007)


oU

UJ-JCQ

00

x:

OBO

-4

0

CQ

03

CO

pd

qd

8d

1

1c

Si

2

— 0000 Tf

OO >Or- —in r-

o o o o

d —p p fno d d

r-; 00 o

d d f*iI I

— d d (5 di <:5 ^1 I _

o r-oo IT)

^ o

o p p --

I I

o o o o oI I

gd

VO OO

d d1

^f ^f r *3 S od d <5

1

Sd1

od

O

d

CAR Vol. 24 No. 3 {Fall 2007)

Measures, Climate of Control, and Performance Measurement Models 957C

om

tu-JCQ<fH

epen

Q

c

o

O

CQ

00

•a

CQ

CQ

30000

d

pd

5d

—' r j OI

qd

O n or - r - i o —

0 0 0o — o —I I

m

— — o o o o o

fn r- p — p p ••d d d d d d ^

p p p — fn — — p p p p pd d d d d d d d d d d d1 1 1 I I I I

—. — O ^ O r ^ j O O —I 1 I

o o o oI I

00 00 r-i vo •^00 — r ] oo fnfn o O O O

— 1^ r j —

8 tN O Oo o o §to o o o o o o o o o

I I I I o o o o o o o oI I

— m o — t n — m o o r ^ — c N O• H 3 C C N r ) O i O O s ^ C N

— — O — f n \ O — —

r*** cn d ^ op p — p p p p

d d d d d d d d dI I I I

O — O O O —

q©pp2588Sddddddddd

^ li

a.u

Is

I



TABLE 6 (Continued)

Notes:

Model and model descriptions are as shown in Table 3.

Only significant, lagged observations of performance drivers might be interpreted ascausally related to performance (shaded rows). Because first- or fourth-differencevariables also contain the contemporaneous value, their coefficients are ambiguousabout causality.

* Variables in shaded rows are hypothesized causes of perfonnance. Coefficients inbold are significant and signed as predicted (t < 0.05, * < 0.01, S < 0.001).

" FR, FRI. and FR3 are highly collinear {R > 0 .70).

CSATand CSATl are highly collinear (R > 0.70).

+ PT01-PT04 and WT01-WT04 are highly collinear (/? > 0 .70).

We use the last three quarters of available perfonnance data (quarters 29-31)to test the predictive ability of the DBSC equations estimated with the earlier 28quarters' data. Table 7 shows that two relations show worse predictive ability withhigher RMSEs and RSSs (dependent variables = PTO and WASG). In contrast, thefull equations to explain customer satisfaction, CSAT, and distributor profitability,PBIT/S. do have 2 and 3.5 percent better predictive ability than their respective,constrained counterparts.'2 The belter predictive ability of the full PBIT/S model,which is an out-of-sample test, indicates that the estimation results for that modelare not sample specific, at least with regard to the impact of sales growth. However,neither improvement in predictive ability is even marginally statistically significantby f-tests (Johnston 1994, 505) of differences in RSSs (a = 0.1). Predictive-abilityand estimation results offer weak support of causality in the PBIT/S equation. (4).No evidence supports causality iti the other three DBSC equations or for otberhypothesized causes of distributor profitability (PTO. WTO. or SAFE).

Alternative models

We also investigate altemative specifications of perfonnance relations. Columns 3through 6 of Table 6 display the estimation results for four alternatives for eachDBSC relation. 3 xhe results of Granger causality tests including fixed distributoreffects, which are binary (0, 1) variables, are shown in column 3. We include distrib-utor effects in an attempt to capture more of the set of "ail available information"and because each distributor might face different market conditions or exert differentefforts. Some of these binary variables are highly significant, but most inference.sabout variables of interest are no more favorable to Granger causality than in equa-tions without these effects. The only exception in column 3 is a significant relationbetween SAFE4 and PBIT/S (p < 0.05). The negative sign suggests that lost-timeaccidents, lagged by four quarters, negatively affect distributor profitability, per-haps through increased insurance costs. We caution that this is an isolated result.

Column 4 reports nonlinear (natural log) transformations of (1) to (4), withoutfixed effects. The nonlinear specification of the WASG model (omitting negative



o

•a

U

a, -o

inen

rroi

u•au

CT1/1

E

uu

ffei

O

•a

1 .21 a-u

— -^ r-; 00

00 — (N —

p r-; rN r-io d o d

pOS

o o o o

m O O f^0 3 - 0 0r-; p — pt6 d d <D

3 '^ Qd d o d

a(3£us3u'•B

uc

, 0

^

00

2cuu

per

u>

L N

egat

i

rN

H30u

IS,

ode

BS?

ausa

li

u

00

ity.

,ccau

•au

B.

t3in

ISE

and

OSUtUS_o>

afo!

ari

>4>

T3

iu

•a

•aebO

IN

o

S -23 o^ s,p 4:

e 5"S

s.a S-

C/U? Vol. 24 No. 3 (Fall 2007)


sales growth observations) indicates a highly significant four-quarter lagged effectof customer satisfaction iCSAT4) on sales growth (p < 0.01). Although it is laterthan company personnel expected, this nonlinear result is consistent with priorresearch by Ittner and Larcker 1998, who suggest that distributors might need toexceed customer service thresholds to affect sales. Because we cannot test forexplicit threshold effects, we regard this nonlinear result with caution. A result inpanel D shows another significant, nonlinear relation between a one-period laggedeffect of sales growth (WASGl) and distrihutor profitability (PBIT/S), but this is aweaker effect (p < 0.05) than found from a four-period lag in the linear Grangermodels. Note that FR4''s significant, nonlinear result in panel A is incorrectly signed.

Finally, columns 5 and 6 report results of one-quarter and four-quarter differ-ences or changes models. Changes models can control for distribulor-Ievel marketand effort effects that might be masked in the original Granger specifications. Thefour-quarter PTO model in panel A shows a significant effect of a correspondingchange in FR (p < 0.05). Similarly, the 1 - and 4-quarter changes models of CSATin panel B show significant impacts of corresponding FR changes {p < 0.05,p < 0.001, respectively), but contemporaneous FR also is significant in othermodel specifications. The 4-quarter change in the PBIT/S model in panel D .showshighly significant effects of corresponding changes in WASG {p < 0.01) and SAFE(p < 0.01). The one-quarter PBITfS change model finds a significant relation withWTO {p < 0.05). These observed effects are inconsistent, but more important, theyare ambiguous about causality because they associate contemporaneous changesand do not cleanly establish time precedence.

Summary of Granger tests and additional considerations

In summary, the time-series data provide some support for cause-and-eftect rela-tions in the DBSC, but the case is inconsistent and not compelling. Several lagged,independent variables are significant "in the presence of all other information", butmost are not. Predictive ability is not established consistently and never signifi-cantly. We find one significant customer satisfaction relation in a nonlinear modelbetween a four-quarter lagged effect of customer satisfaction iCSAT4) and salesgrowth (VMSG).The only support for cause and effect across multiple modelspecifications appears in a tour-quarter lagged effect of sales growth (WASG4) ondistributor profitability (PBIT/S) in several linear Granger models. However, thisstatistical significance in the model is accompanied by insignificantly improvedpredictive ability. The case for cause and effect in the DBSC overall is quite limited.

Distributors' performance on DBSC measures exhibit signs of conformity tocompany targets. Panel A in Figure 3 presents the time series of proportions of dis-tributors' target performances for CSAT The proportions of red, yellow, and greendistributors obviously shift to green performance over time. We also visuallyexamined regression model residuals for evidence of the development of expectedrelations over time. Panel B in Figure 3 presents an error-bar chart of the pertbrm-ance time series for customer satisfaction (CSAT) residuals from a linear regressionof (2) using the full 31 quarters of data. The early time series of CSAT regres-sion residuals is noisy but reflects obvious tightening and overall improvement in


Measures, Climate of Control, and Performance Mea.surement Models 961

the last five quarters. These results show that the DBSC is related to impacts onperformance despite lack of demonstrable cause and effect.''' Figures for other per-formance measures are similar.

Our multiple tests find that the DBSC has limited significance and predictiveability, which refutes cause-and-effect relations in the DBSC as an explanation forits continued use. This apparently flawed model could preclude reliable prediction,decision making, learning, and communication. Yet distributors' DBSC performancehas improved, and the company has continued to use the DBSC in subsequent

Figu re 3 Time series of customer satisfaction perfonnance, Q1-Q31

Panel A: Customer satisfaction, percent red/yellow/green, all distributors

100% •

90% •

80% -

70% -

60%

50% -

T " r " I 1 1 1 1

1 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 2H 29 30 31QuHrier

Panel B: Error-bar chart of customer satisfaction perfonnance, all distributors

EC

SAT

vi

j_

Mea

n

0,95 -

0.90 -

0.85 -

0.80-

0.75 -

0,70-

0,65 -

I 4T 1 T " '

-9-

1—1—1—1—1—1—1—1—1—1

^ X

1r—1—1—I—1—1—1—I—1—• r • r' i—i—i—i—1—1—1—1—1—1—1—

1 2 3 4 5 6 7 9 10 1112 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31Quarter


962 Contemporary Accounting Re.search

periods. In fact, the company has placed more weight on the DBSC for organiza-tional change and variable compensation of distributors and is deploying theDBSC to its worldwide distribution channel.

Finding evidence to support cause and effect within PMM might be possible,but not easy; furthermore, such evidence can only conservatively refute cause andeffect (Popper 1959, 1963). In the context of PMM, at least three reasons workagainst establishing Granger causality: (a) managers adapt the firm's actions andthe underlying production function to PMM and other feedback (hence, statisticsare unstable); (b) a PMM that is not a fully specified input-output model may notreflect underlying cause and effect sufficiently; and (c) cause and effect might notexist in nonphysical (portions of) PMM (for example, relations of service perform-ance). This study's empirical findings, which are either contrary to normative theoryor reflect a PMM that cannot exhibit cause and effect, motivate our continuing thestudy. In the ca.se of the enduring DBSC, explanations other than cause and effectare required. Hence, we believe there are at least two theoretical explanations as towhy a PMM can endure without evidence of cause-and-effect relations: (a) mis-specification of DBSC relation types and (b) an incomplete theoretical framework,

5. Reconceptualized theoretical framework

As amply discussed previously, the DBSC does not pass rigorous Granger causal-ity estimation and predictive-ability tests, but the DBSC is an enduring PMM at asuccessful company. We do not accept that managers' beliefs about the DBSC areeither irrational or deceptive, and our failure to find evidence of cause and effectchallenges our original beliefs about whether cause and effect exist or are neces-sary in the DBSC. These results lead us to reconsider the nature of the relations inthe DBSC that were previously published (Malina and Selto 2001, 2004), Othertypes of relations can and likely do exist in the DBSC and probably in otherPMMs. An expanded analysis of relations made us realize that logical and finalityrelations also can exist among measures in PMM. Importantly, these other rela-tions are consistent with managers' continued use of the DBSC for managementcontrol. The classifications of relations reflect more than semantic differences; thedifferences have important implications for PMM development, validation, use,and feedback. We discuss logical and finality relations, then we reanalyze ourDBSC results.

Logical relations

Logical relations exist by buman construction or definition and may be commonelements of PMM. They are tbe results of related human constructs, such as math-ematics, language, and accounting (N0rreklit 1987, 164; Ijiri 1978, chs. 4 and 5),Logic, for example, defines that debits equal credits and, in general, logic is a con-sistent tool for creating and managing human reality. Financial and managementaccounting systems, DuPont models (return on investment [ROI]), and net presentvalue calculations are common examples of logical models that measure economicprofitability. Although specific applications often vary, tbe logical relations ofthese models are independent of firm-level contingencies. In accounting, the effect

CAR Vol. 24 No. 3 (Fall 2(X)7)


of an action on profit (for example, sale of a product with a positive contributionmargin) necessarily occurs by the double-entry logic of the accounting system, notcause and effect. Note that the relation between two phenomena cannot be bothlogical and causal. In a cause-and-effect relationship, the cause happens before andindependently of the effect, and the cause must be logically independent of theeffect.

It is a social fact (Searle 1995) that financial accounting models are used inour society to tneasure and evaluate the financial performance of a firm. Thisimplies that financial analysis is needed in a firm to structure and evaluate the eco-nomic a.spects of decisions and actions. For example, the creation of profitabilityby making customers loyal depends on the revenues and costs of making themloyal, which dictates that we have to use financial analysis to evaluate whether aloyal customer is profitable (Norreklit 2000). Therefore, any financial logic embed-ded in the PMM has to be linked to the rules of financial accounting performance(N0n'eklit, Norreklit, and Mitchell 2007), not cause and effect.

Logical models, such as accounting, are not refutable by empirical evidence,only by deductive reasoning. For example, decomposed DuPont relations, .such asROI equals return on sales multiplied by asset tumover, are logical, not cause-and-effect relations. A regression model of these logically related variables does notgenerate empirical evidence on the validity of the logic or formula. Statistical sig-nificance, or the lack thereof, speaks instead to the reliability of ceteris paribusconditions that support the logical relation, such as control of pricing and costs andother related but omitted logical links. In practice, many activities logically influ-ence profitability, but PMMs appear to be simplified combinations of KPI, notfully specified accounting models. Inevitably, logical ceteris paribus conditionswill be difficult to control or observe in actual PMMs. and statistical explanation ofrelations among logical KPI will be less than perfect, perhaps insignificant.

Finality relations

A finality relation exists when (a) one believes that a given action is the best ormost desired means to an end, and (b) the belief, desire, action, and end are relatedby custom, policy, or values (Arbnor and Bjerke 1997). Actions driven by finalityare performed because the actions conform to the beliefs and wishes of a person(or group). Acceptable outcomes (for example, profitability) can reinforce thesefinality relations, but cannot transform finality into cause and effect. Finality isfundamentally different from cause and effect because finality-driven actions andoutcomes are not independent or uniquely observable (Mattessicb !995). They areconfounded and violate Hume's first criterion of independence of phenomena. Fur-thermore, observation of subsequent favorable outcomes reflects the results of anengineered process, but does not signal a generic process to that end.

Finality relations have other characteristics that set them apart from cause andeffect. Unlike cause and effect, any chosen means is but one of several or manythat can be u.sed to reach the end. Furthermore, a finality relation can be idiosyn-crdtic to a particular .setting or context (Arbnor and Bjerke 1997, 176). For example,corporate vision and mission statements commonly contain finality relations.

CAR\o\.2A No. 3 (Fall


Contingency theory is a common academic expression of finality in organizations,and empirical test.s of contingency concepts often do not generalize beyond a spe-cific firm or sample (Drazin and Van de Ven 1985; Chenhall 2003). The oft-citedroie of a BSC to tell the "story of the company's success" is another expression offinality that directs employees to preferred actions that might not be generalizablebeyond the specific company and time. Finality relations often rely on incompletearguments where premises are lacking, such as unspoken, ceteris paribus condi-tions that are nearly impossible to control in natural settings. This complexity ofrelations, in conjunction with a lack of independence of phenomena, is an indicationof finality rather than causality. However, to use finality relations to achieve sustainedcontrol of actions, a finality belief that a given action leads to an end must be reliableor perceived as such, at least in a specific context. In many practical situations ofmanagement control, finality and logical relations work tightly together when oneuses financial analysis to decide on strategies and policies, similar to Simons'sbelief-system controls (Simons 2000,276), For example, ceteris paribus conditionsthat exclude unprofitable products and customers might engineer a reliable relationbetween customer satisfaction and profitability.

Statistical analysis might be helpful to establish context-specific reliability ofa finality relation, but it cannot be definitive. Validating a finality relation as thebest or unique means to an end is complicated by equifinality and finite data.Although statistical validation may not be possible, financial analysis of costs andbenefits of finality-driven strategies might explain their use and longevity despitestatistical insignificance of finality relations among measures.

In summary, statistical tests, such as Granger causality, are appropriate for val-idating cause-and-effect relations, which may be uncommon except in PMMs thatreflect physical productive processes. In contrast, statistical tests are irrelevant forestablishing the validity of logical relations and may be insufficient for finalityrelations, both of which may dominate most PMMs. Furthermore, feedback fromfinancial analysis of logical and finality relations may explain the duration andevolution of PMM to a greater degree than statistical analysis. Thus, our generallack of statistical support for the enduring DBSC refiects the presence of logicaland finality relations among financial and nonfinancial performance measures.Company support for the DBSC may reflect its favorable impact on companyprofit, which results from a tangled chain of financial logic and finaiity that is notobservable at the distributorship level and might be exceedingly difficult to discernat the company level.

Reanalysis of the data from model to results

Our previous beliefs about cause and effect in the DBSC were based on normativeassumptions and qualitative analysis, which, like other empirical methods, is sub-ject to researcher bias. Hence, we expecied cause-and-effect relations, and wefound them. However, the statistical results and our more refined understanding ofrelation types in PMM challenge those prior beliefs, which were reinforced by theoriginal qualitative analysis. If we had approached the qualitative data withbroader, less dogmatic beliefs about the nature of PMM relations, perhaps we



would have reached different conclusions about the prevalence of cause and effectin the DBSC and the applicability of Granger causality tests to this case.'"^

With a wider theoretical lens, we recoded the original and 2005 qualitativeinterview data by asking:

1. Does this relation reflect independence of phenomena, time precedence, andpredictive ability?

2. If so, code the relation as cause and effect.

3. If not, code the relation as logical or finality, as appropriate.

Reanalysis of the qualitative data reveals no unambiguous, cause-and-effectrelations, often because of violations to tbe independence criterion. The DBSC'slogical relations of financial cost-benefit are now obvious, but we had coded thempreviously as cause and effect. The DBSC contains an inventory replenishmentrelation (I) plus familiar relations between inventory tumover and distributor prof-itability (4), and relations between revenue and cost drivers and profitability (4).All are logical relations, not cause and effect, because of their derivation from theaccounting system. Similarly, we now identify the customer satisfaction (CSAT)relation with fill rate (FR) as a finality relation because having parts available ontime is the company's preferred action to increase customer .satisfaction, but that ishardly the only approach. Likewise, the relation of customer satisfaction (CSAT)driving sales growth (WASG) most likely is finality, not cause and effect because ofthe numerous ceteris paribus conditions required. We now classify all DBSC rela-tions as logical or finality, and none as cause and effect, as shown in Table 8.

Let us revisit the four DBSC equations, which are abbreviated below. Othervariables (generically symbolized by Z,) are proxies for "ail available information"and, as before, are not of direct interest to this study.

) (la),

, ,Z,) (2a),

WASG, = HCSATi, Z,) (3a).

PBIT IS, = kiWASG,. PTO,. WTO,. SAFE,. Z,) (4a).

Underlined variables represent five logical relations; bold variables represent twofinality relations. We explain and illustrate our revisions more fully as follows.

Logical relations

The relation of parts turnover (PTO) as a function of fill rate to customers {FR)(la) is a relation that derives from the logic of inventory replenishment, but otherceteris paribus conditions surround this logical relation. For example, the expresseduncertainty about fill rates from the company can induce distributors to build

CAR Vol. 24 No. 3 (Fait 2007)


T3CQ O

EU

Q

o

1co

u

oOBC

man

§aE0U

o

UJ

O — 00 Os

O O D >.O

o o — —

o o c

„ . .*-• ta

« .S o oU £ -J P

o

1

IkDA

UJ


Measures, Clitnate of Control, and Perfonnance Measurement Models 967

inventory levels to ensure favorable fill rates to customers. This problem was iden-titied by most distributors. Consider several distributors" explanations:

As we are customers of the factory, [fill rate] is very imporlant to us. If wearen't receiving a high till rate from the factory, we can't achieve a high tillrate to our customers. It's a domino effect. The factory is having availabilityproblems now ... If one piece of the |distribulion] channel breaks down, all thepieces are greatly affected. (Distribulor D)

What about our fill rate from [the company]? Big interaction there ... Our fillrales are always higher than theirs. Theirs is 61% to us and ours is 90% to cus-tomers. We have to stock more inventory than them. (Distributor H)

Distributor E explains the logical impact of inventory tumover on distributorprolitability (4a).

Obviously if you have less inventory and you still have good availability [ofparts to customers] then you'll have more cash available, and less expensewhich will make you more profitable. (Distributor E)

The logical impact of safety on distributor profitability (4a) is also evident incomments such as:

It's more costly after the fact than it would be to build it [safety] into the pro-cess and show where it fits into the cost of doing business. If you look at workerscompensation cost, the cost of medical care today, and injury intervention, allof those things come off the profit side of the business. (Manager N)

The relation between weighted average sales growth (WASG) and increase indistributor profit (PBTT/S, {4a)) likewise is a logical relation of financial cost-beneht.The results in panel D of Table 6 show a consistently significant logical relationwith current weighted average sales growth {p < 0.001) and with a four-quarterlag {p < O.O0I)."*The consistent significance indicates that this relation must betightly controlled; that is, increased sales of profitable products to profitable cus>tomers drive profits when key ceteris paribus conditions are maintained. A recentinterview with a senior executive of the company confirms that the company con-trols conditions in the relations between sales and profit. Top management hasdecided which products are most profitable (to the company) for a distributor tosell, and it limits distributors' profitability by setting minimum product price mark-ups, which appear to hold. The lagged effect means that it can take approximatelyone year for an average new customer to become profitable to the distributor.Although customers might be profitable immediately to the company, because ofthe company's control of products and prices, the costs of extra services andcustomer development borne by the distributor appear not to pay off quickly. Dis-tributors, of course, recognize that they must absorb these costs.



They have not given us any tools to sell Ihe product over the competition. Thisis a price sensitive market, and we're holding the line on our prices, and we'renot giving away incentives like our competitors. They need to adjust the [sales]target if they aren't going to help us. (Distributor A)

[The company] is not won^ying about what it is costing distributors to improve.They are looking at their cost. (Distributor G)

Finality relations

We recoded some relation.s as finality rather than cause and effect. For example,the commonly voiced argument that follows describes a complex finality relationthat involves achieving high first-time parts fill rate (FR) to customers as one wayto improve customer satisfaction (CSAT, (2a)) and that must require many controlsto be valid.

The measure (FR) is important and quite valid ,.. It is a direct measure of howwell we serve our cu.stomers. If we are doing 99 percent, we are only disap-pointing 1 percent of the customers, h is a valid measure because it tells ushow we are doing In giving the customer what they ask for the first time. Peopleare very sensitive. They let us know if we're nol living up to expectations.Some of our dealers are looking elsewhere to get parts because of the stocking|fill rate] problem. (Distributor A)

Clearly, the fill rate is an important measure, but the relation to customer satis-faction cannot be cause and effect. A consistently significant contemporaneousresult in panel B of Table 6 (/? < 0.01) does support respondents' strong finalitybelief that a higher fill rate is associated with higher customer satisfaction. How-ever, it is impossible to determine whether other factors not included in the PMM,including other dimensions of service quality, are driving this result. The relationalso appears to be idiosyncratic to this company's preferred approach, becauseother means to improve customer satisfaction surely exist (for example, lowerprices, fewer processing mistakes).

A finality relation also exists between customer satisfaction (CSAT) and salesgrowth (WASG, (3a)). At the operational level, increased customer satisfaction isnot free but may increase sales. Sales growth also can be affected by uncertain fac-tors that might not be controllable by distributors. These include competitors'actions, industry changes, and changes in customer values and tastes. The resultsin panel C of Table 6 indicate that this is not a reliable finality relation, becauseonly one significant result is found across the five model specifications (one of 14coefficients). '^ Either customer satisfaction as a driver of sales growth is an invalidbelief or macroeconomic factors like sales prices or industry effects influence tberelation but are not controlled.

In light of the reclassified DBSC relations, the weak Granger causality resultsreported earlier are no longer surprising. Logical relations cannot be validated orinvalidated by the statistical tests. The DBSC's far from perfect R^ can be attributed

CAR Vol. 24 No, 3 (Fall 2007)

Measures, Climate of Control, and Perfonnance Measurement Models 969

to lack of control of important ceteris paribus conditions, such as distributors'response to the company's fill rate. The finality relations have incomplete reliabil-ity, which may reflect a dynamic environment and adaptive behaviors.

6. PMM and the climate of control

Cause-and-effect relations among measures appear important for prediction, deci-sion making, communication, learning, and goal congruence. Certainly cause andeffect might exist in some PMMs. In this case, however, the company's reliance onthe statistically weak DBSC leads us to consider whether cause and effect are nec-essary to the success of a PMM. The reinterpretation of DBSC relations as logicaland finality relations does give us a better understanding of the weak statisticalresults obtained, but it does not by itself provide a convincing argument for whythe company continues to support the DBSC and plans to expand its use. and whydistributors increasingly conform to performance targets. It is possible, for exam-ple, that the threat of the loss of the distributorship contract is sufficient to coerceconforming behavior. However, voluntary turnover other than retirement amongthe distributors is almost unheard of at this company, and most distributors agreewith the intent of the DBSC (Malina and Selto 2001). Both indicate a mutuallybeneficial relationship. We posit that an organization, like the one studied here,may use a PMM to reflect and reinforce a "climate of control" that reflects thecompany's environment, style of management, and institutional and social cultures.Furthermore, the climate of control achieved and reflected in financial successexplains the longevity of PMM.

Contingency research in management control (e.g.. Abernethy and Lillis2001) recognizes the importance of "fit" between strategy, culture, managementstyle, uncertainty, and performance measurement as important to the design andeffectiveness of control systems. Thus, we posit that intended fit influences thedesign of PMM such as the DBSC. We further posit that beliefs about relationsamong strategically important variables influence managers to create PMM withlogical and finality relations that are supported by financial feedback and cause-and-effect relations, which may be validated by statistical tests. Uncertaintiesabout these relations may contain much of the uncertainty construct that contin-gency research often measures poorly. A firm may install a PMM that reflects itsclimate of control to communicate its strategy to enhance learning and thelegitimacy and fairness of goals and performance measurement. The aim of thisPMM-based communication would be to increase motivation and to improve deci-sions and financial success (e.g., Anthony and Govindarajan 1998, 7, 95). Wereason, therefore, that a firm might regard a PMM as effective if it contributes togoal congruence and desired conforming behavior, reinforced by improved finan-cial performance. For example, top management at the company confirms that acontrol environment of ceteris paribus conditions for sales and profitability aremaintained in the company. We posit that the DBSC enhances the company's cli-mate, which conspicuously features pay-for-performance and result control(Malina and Selto 2001, 2004). Furthermore, the DBSC affects motivation andconformity favorably if the means of perfonnance measurement are regarded as

CAR Vo\.24 No. 3 (Fall 2007)


legitimate and fair. These elements of climate of control are illustrated in Figure 4.We next discuss the elements of otir proposed theory of PMM effectiveness in thecontext of the DBSC.

Pay-for-performance culture

As discussed earlier, distributors appear to have ample reasons to regard the DBSCmeasures seriously. Both variable compensation (now about 50 percent of totalcompensation) and contract renewal depend on DBSC performance. The DBSC isthe foundation for the pay-for-performance climate (Miller and O'Leary 1987)created for the distribution channel. A few DBSC measures are controllable (forexample, safety) by distributors, but many are influenced by less controllablefactors. For example and as described previously, a distributor's parts fill rate tocustomers and its inventory policy are affected by the company's parts fill rate to thedistributor. Therefore, the company uses the DBSC for relative performance evalu-ation (RPE) by ranking and comparing distributors by DBSC performance. Thepay-for-performance eifects on motivation are readily apparent in these representa-tive statements from the interview data.

Figure 4 Model of PMM control effectiveness theory

Management style,strategic goals.

accounting tools

Performancemeasurement model

(design and use)

Beliefs aboutperformance

measure relations(uncertainty)

Business modelcommunication

(leaming,legitimacy,fairness)

Goal congruence(motivation,conformity)

Types of relationsbetween measures

Logical and/orfinality

(engineered)

Cause and effect(observed)

Business modelpredictive ability

Decisioneffectiveness

Statisticalfeedback

Financialfeedback

— "

Financialfeedback

Note:

Intermediate feedback loops also can and probably do exist.



No one wants to be #31. They are very competitive people. (Manager L)

If a distributor is in the bottom quartile for 2-3 quarters in a row, then they areon probation. (Manager J)

We are competitive. Anytime you publish a report and there [are] 31 entitiesbeing measured using the same metric, it matters what rank you are. Even if noone looks at the rank, I want to be #1. (Distributor E)

Results-oriented control system

Interview data show that the DBSC was designed as a result control. Merchant's1998 four conditions for effective result control are knowledge of the result desired,controllability of the desired result, measurability of the controllable result, andperformance targets. Although controllability varies across measures, the DBSCaddresses these conditions and focuses distributors on results that benefit the com-pany. Consider the following selections from many similar quotations:

[The DBSC] is a way to measure them [distributors] in a balanced way. whatthey are really responsible and accountable for. It provides appropriate weightsfor what you want them to do. The company will benefit because you haveattached certain weights that you want to drive them to perform well in. Theweights make them swing that way. It's driving behavior toward the higherweights. (Manager K)

It's extremely and painfully obvious which are the most important [results]. Ifyou're the worst in [X] market share, you can't overcome il by greens in otherareas. That's the lifeblood of the company. (Manager L)

When [the company] added new measures that they didn't tell us about and thendiey were red, it's not a subtle sign that we need to look at that area. (E>istnbutor F)

Legitimacy

Although the company did not use outside consultants, it prominently named itsmodel the "Distributor Balanced Scorecard" and used Harvard Business Scbool-educated employees to design it. Although some distributors regarded the DBSCwarily at first — especially noting its early lack of "balance" — none openly chal-lenged more than small parts of it. Even if the PMM is always a "work in process",an organization can use PMM to build legitimacy by projecting rationality and effi-ciency to intemal and external constituents (Camithers 1995; Meyer and Rowan1977). Most distributors accepted the model and its norms as legitimate. A com-mon sentiment was:

I like all A's on my report card, so I want all of them green. I agree with almostall the measures. They are indicative of where you are. (Distributor F)

Interestingly, neither the distributors nor the company had conducted statisticalanalyses to validate the DBSC. However, they observed relations between per-



formance measures, such as that between fill rate and customer satisfaction. Thisreinforcement of beliefs also adds to the legitimacy of the DBSC. For example,consider this almost unanimously expressed belief:

[Parts fill rate] measures whether we have the right type of inventory parts onhand and the right quantities. It's one of the most important measurements wehave here. The key thing is the right product mix and quantity and to satisfythe customer the first time around. (DisU"ibutor D)

Fairness

Prior to the DBSC, managers and distributors acknowledged that subjectivity andfavoritism affected management of the distribution channel. The DBSC wasintended to make evaluations and evaluation processes appear more fair and objec-tive (e.g.. Bumey. Henle, and Widener 2006). Managers may accept a PMM if itpersuasively builds on the ideas of the market economy and "fair contracts", whichgovern social relationships in the United States (Bourguignon, Malleret, and N0r-reklit 2004). The idea of faimess expresses the opportunity open to everyone towork their way from the bottom to the top. Everyone is expected to act freelyunder contracts to which he or she chooses to be committed and under a generalmoral claim to faimess. Furthermore, faimess is associated with suitable remuner-ation for a person's work perfonnance and with the equal treatment of everyone(d'lribame 1994). Consider the following representative quotations from distributors:

IThe DBSC| is intended to be a way that the factory can measure the perform-ance of (the! distributor network in such a manner that it puts everyone on alevel playing field as far as measures. (Di.stributor D)

As [the companyl did ihe every-3-years cotitract review. I had heard that therewas speculation that some guys got an easier or harder approach based onwhether they were friends or enemies of [the company]. |The DBSC] at leastgave some quantitative basis to the evaluation process. It's more objective ^ldblack and white on key areas. (Distributor F)

I grew up working for a CPA and he ingrained in me that if you can't measureit, you can't improve it. I like this because its measures I already have andbecause it takes some of the guessing out of "how does |the company] viewme?" I just like knowing my grades. I assume that if I have a green, |the com-pany] is grinning. [The DBSC] helps me think that greens will take the stressaway for the next contract review. (Distributor F)

Motivation and conformity

On the basis of getting what one both measures and rewards (i.e., Merchant 1998),one expects that distributors will manage their processes (and perhaps the mea-sures) to achieve favorable performance ratings. Even in the absence of demonstratedreliable measure relations, acceptance of the system and conforming behavior will

CAR Voi. 24 No. 3 (FaU 2007)


also be consistent with a DBSC that has as a primary purpose the creation of a cli-mate of control through pay for perfonnance, fairness, and legitimacy.

Archival performance data show conformity to norms over time for manyDBSC measures. The company has merged 11 distributors over the past two yearson the basis of DBSC results and a desire for better overall distribution efficiency.Although the mergers have created periods of performance instability, genera!Improvements in DBSC results for most measures are apparent. For example, thepercentages of distributors attaining the green (highest) scores on heavily weightedcustomer satisfaction have increased over time, while those distributors with yellowand red scores have decreased over time (see Figure 3).

Climate of control summary

The model of PMM effectiveness that we propose is in Figure 4. ** Althotighcause-and-effect relations seem desirable, they may be unnecessary or infeasible ina highly uncertain, dynamic environment. Even in stable conditions when cause-and-effect relations are indicated, as a practical matter any observed lack of statisticalreliability may be attributed fo a PMM's continuing evolution. As long as fhe organ-ization is committed to achieving a reliable PMM in the future, a PMM could be anotherwise effective control device despite ifs current lack of statistically reliablerclalions among measures. More often, perhaps, PMM will contain logical andfinality relations that can support the desired climate of control. By designing aPMM to be a result control and pay-for-performance tool, and by establishing itsfairness and legitimacy, management can motivate employees to conform to com-pany expectations. Thus, the climate of control might be sustainable even whenperfonnance relations cannot be unambiguously or statistically demonstrated ascause and effect. This climatic role for PMM mighl outweigh a PMM's usefulnessfor prediction and decision support. In an uncertain environment, the rhetoric of abalanced scorecard model combined with face-valid measures, valid logical rela-tions, credible finality relations, and positive financial feedback may be sufficientfor a PMM to be considered successful.

7. Conclusions, limitations, and future research

Conclusions

Cause-and-effect relations among perfonnance measures have been argued to beessential features of PMMs because they can aid financial prediction and decisionmaking as well as create effective leaming, communication, and goal congruence.We approached this study with the intention of testing the validity of cause-and-effecf relations in an enduring PMM at a Fortune 500 company. The companyestablished a distributor balanced scorecard for its distribution channel. Qualitativedata from interviews with managers and distributors prior to statistical tests arereflected in perceived relations among the DBSC measures, a finding that estab-lishes face validity for the model tested statistically (see Malina and Selto 2001.2004). We evaluate the DBSC for evidence of Granger causality, but find at bestlimited support for any cause-and-effect relations both in initial and expanded


974 Conlemporary Accounting Research

time-series data sets. Our statistical results point to explanations that we could notaccept — that the DBSC must be a fad or a deceptive exercise of managementpower — because the DBSC has endured and worldwide deployment is planned.Statistically unreliable relations thus far have not been a barrier to continued andmore confident use of the DBSC in the North American distribution channel of thislarge, successful, international firm.

This dissonance motivates a review of the types of relations that can appear inPMM. and this broader review identifies two other types of relations in PMM, logicaland finality relations, that can complement or might supplant cause-and-effectrelations. Without a proper understanding of the different types of relationships, adeeper understanding of the design and use of PMM might not be possible. Forexample, any PMM relation involving financial measures of performance reflectsaccounting logic that cannot be refuted by empirical evidence. The different rela-tions combined with a further analysis of both qualitative and quantitative data leadus to conclude that cause-and-effect validity might be less important to some con-texts than a PMM that is perceived to be legitimate and fair and that supports aneffective climate of control. Our careful use of theory both to motivate the cause-and-effect study and to interpret the results indicates that justifying PMM only onthe basis of valid cause-and-effect appears to be myopic in this case. Hence, thisstudy indicates that one should not reject the validity of a PMM simply becausestatistical evidence of cause and effect is lacking. Organizational validity may heelsewhere, as summarized in Figure 4. Whether and when a PMM successfullysupports an effective climate of control without intended or validated cause-and-effect relations deserves future research.

Previous studies (e.g., Malina and Selto 2001) have concluded that PMMs canbe effective strategy communication and motivation tools. The present study indi-cates that the DBSC also serves as a useful and effective result control through theuse of pay for performance and perceptions of fairness and legitimacy that createmotivation and support conformity. Measurability of performance and setting per-formance targets can be helpful to establishing a climate of result control, but"softer" considerations such as the perceived fairness and legitimacy of the PMMalso appear to be important to its effectiveness as a result control and pay-for-performance system. The perceived relational properties found here, combinedwith other attributes of this PMM and acceptable feedback from financial success,appear to be sufficient to support continued PMM use.

Limitations

Our study has failed to support the normative assumption of cause-and-effect rela-tions in a PMM at a business-unit level. At a minimum, our study has refuted causeand effect as an explanation for the continued use of this company's DBSC (Pop-per 1959, 1963). As in all case research, one can question the reproducibility of theresults, but statistical support for cause and effect will be elusive in the best of circum-stances because of incompleteness of PMM, managers' adaptations to feedback,and instability of firms' production functions.



This study is limited by the quarterly data that might disguise shorter responsetimes among leading and lagging performance measures. Some measures oncethought to be important to the performance model were dropped by the companyfor measurement deficiency reasons. These omissions might cause material bias inestimated statistical relations, if in fact they are important to explaining overallperformance. The data are limited to the distribution channel of the company'svalue chain, but overall profit accrues to tbe entire chain. Thus, distributor profitabil-ity, which is tightly controlled by tbe company, might not reflect the full distributioncontributions to overall profitability.

Future research

We suspect that archival PMM data from most organizations will be similarlymessy for several reasons. First, thorough research and development of PMM meas-ures migbt be impractical given the strategic urgency of implementing a newPMM. Learning by doing and continual improvement seem likely. Second, stra-tegic and operational changes will occasion changes in tbe PMM. Third, oneshould expect firms to take actions based on PMM results to improve tbe organiza-tion, which will change tbe data-generating processes. Because all of thesechanges to tbe production function and interruptions to the time series of data arelikely in dynamic organizations, one should not expect anything like laboratoryconditions and measurements. If a firm intends and even achieves a causal PMM inthe real world of dynamic organizations and periodic data collection, cause andeffect migbt not be observable or testable. Thus, tests for cause and effect may notbe useful for judging even intentionally causal PMM.

We acknowledge tbat challenging earlier results with critical argumentationand otber types of data are important for advancing our knowledge of these phe-nomena; however, a study's methodology must fit tbe nature of the problem (Popper1961, 1963; N0rTeklitet al. forthcoming). If the logic of financial accounting formsa crucial part of a PMM, one must design an empirical study to reflect tbe businesslogic of tbe company. Altbougb logical analysis can refute logical arguments, wecaution that wbile qualitative analysis is suggestive it may be insufficient by itselfto support or refute empirical hypotheses. Thus, we believe that dialogue-basedresearch methods complement statistical tests of cause and effect.

We agree with Ittner and Larcker 2003 tbat firms and researchers shouldexamine PMM relations between means and ends and carefully estimate tbe financialconsequences of alternative actions. Only the rare firm living in a stable environ-ment may be able to establish a predictable, cause-and-effect business model.Because for most firms tbe business context is dynamic and does not followmechanical laws, firms may intentionally, but perhaps without regard to labels andtheir implications for validation, create PMMs that cannot be validated statistic-ally. Thus, estimating effects and predicting future performance of logical andfinality relations or changing cause-and-effect relations must depend on more tbanextrapolations of prior results. Not only past results but also tbe financial impactsof future opportunities should fonn part of performance prediction, and inevitablymanagement must make subjective assumptions and judgements. Evaluating the



validity of PMM may require logical, qualitative, and financial cost-benefit analy-ses (including business-model simulations); the statistical tools of normal .sciencemay not apply easily.

On a practical level, more work might be justified to improve existing PMMmeasures and accuracy of reporting and to reconfigure PMM as the organizationgains experience and expertise. Consistent commitment and fine-tuning mightimprove its statistical reliability and predictive ability over time (e.g.. Shields andYoung 1989), particularly in PMMs that reflect physical processes and possibly forfinality relations such as those involving customer satisfaction. However, if logicaland finality relations are relatively frequent, financial, cost-benefit analysis will bemore important to judging the reliability of PMM than statistical analysis. It is pos-sible that companies care more that the PMM tells an intuitive story and providesan accepted and effective basis for result control than whether the PMM embodiesstatistically significant relations throughout. In the case studied here, for example.,the firm might focus on establishing the faimess and legitimacy of the DBSC in itsforeign distributorships before deploying it globally. The firm also could investi-gate the financial cost-benefit behind the consistent logical result that distributorprofitability lags sales growth by a full year. Perhaps seasonality drives this lag, butperhaps the company and its distributors could leam how to make their new cus-tomers profitable more quickly.

On the basis of the summary of relations shown in Figure 4, we pose the fol-lowing "climate of control" propositions for consideration by future research:

PROPOSITION 1. An organization's climate of control influences the design ofPMM.

• Factors include management style (for example, pay for performance).strategic goals, and the use of accounting tools.

• Performance measure relations in PMM are functions of contingenciessuch as desired climate of control and environmental uncertainty.

• Climate of control and beliefs about relations among performancemeasures interact to affect the design of the PMM.

PROPOSITION 2. The design of a PMM affects its use.

• Business model communication is moderated by the types of relationsimbedded in the PMM.

• Business model communication generates control legitimacy, fairness,and learning that affect motivation, conformity, and goal congruencewithin the organization, moderated by the business modeis predictiveability.

• Business model predictive ability is moderated by the types of relationsimbedded in the PMM.

• Business model predictive ability and goal congruence affect decisioneffectiveness.



PROPOSITION 3. PMM design and use are influenced hy financial feedbackbecause all elements of the proposed climate of control theory are dynamic.

Although PMMs such as the BSC have spanned the globe and appear in everytype of business, government, and nongovernmental organization, we have muchto learn about how complex PMMs are used. Future research also can investigateconditions where logical and finality relations are expected to complement or sup-plant cause-and-effect relations, or vice versa. We have witnessed what appears tobe substitution of finality and logical relations for cause-and-effect relations in apredominantly servlce-orienled PMM. Whether this is intentional or common isunknown to us, but we suspect that many, perhaps most, PMMs will tend lo havefew unambiguous cause-and-effect relations. Cause-and-effect relations might becommon in the PMMs of organizations that are strongly based on physical pro-cesses, such as those in extractive and manufacturing industries. Service-orientedorganizations or those parts of large organizations that are largely service, itappears to us, may be far more likely to construct PMMs with finality relations.Logical relations that link upstream outcomes to financial outcomes may beequally likely in all types of organizations. Given that companies operate in a con-text of accounting performance, we know that the measurements have to be linked10 financial performance one way or another, but we do not know much about howthe links are constructed and made operational. We also do not know whetherPMM success and ultimately organizational success are positively associated withcomplementary use of all types of relations or whether focus on one or anotherincreases PMM success. We look forward to future research to further examinethese issues to better our understanding of this complex phenomenon.

Kndnotes

1. Frigo (2(X)2a. b) is representative of the widespread belief among practitioners Ihat the"proper" KPis are related by cause-and-elfect relations to measures of financialperformance.

2. Forrester (1994). summarizing the then mature field of systems dynamics, also hasargued for the value of linked systems models of performance.

3. See Malina and Selto 2001. 2004 for extensive descriptions of the research site,original interviews, and qualitative method used.

4. Ititerviewees discussed several other perfomiance measures that at the time of thisresearch did not have suf^cienl data to support statistical tests that are discussed later(for example, service cycle time, which was believed to be a driver of cu.stomersatisfaction). For consistency with later statistical analyses, this study addresses themeasures that were used for the entire time series.

5. Ninety-five additional comments referred to vague relations between one DBSCmeasure and other, unspecified drivers; for example, "there are other measures thatdrive [financial measures]".

6. Ambrosini and Bowman (2002), Malina and Selto (2001). and Friese (1999) are amongthe studies that use the relational data base feature of qualitative data software to buildrelational maps.



7. The regression residuals are not importantly (''m.u = 0.055) or significantly correlated(a - 0.05) across equations, which permits the use ot" ordinary lea.st squares (OLS)(Bollen 1989,64,404). Kolmogorov-Smimov tests do not rejeet hypotheses that theprediction errors are normally distributed (a ^ O.OI). These and other untabulatedresults are available from the authors.

8. We are unable to cleanly analyze only the 19 continuing distributorships for the entiretime series because post-merger data is consolidated. Analyzing data for only the 19survivors during the pre-merger time period. 1997 Ql-2004 Q3, generates results thatare less favorable to Granger causality than reported here.

9. Initial descriptive analysis shows that WASG has a large range for a propotiionalmeasure. Further investigation reveals that two distributors entered new markets earlyin the time series and had exceptionally laiê percentage sales growth in those marketsin the first year, growing from a near-zero base. All reported results retain the fiveoutlying observations of WASG from these two distributors; omitting theseobservations slightly improves the significance of several tests involving WASG, butdoes not affect results for other tests.

10. Chow f-tests using identical data sets show that three of the four full models (explainingCSAT, WASG. and PBITIS) have statistically superior explanation {p < 0.05)compared with constrained models. However, only one of the improvements in full-model explanations is driven by a lagged driver (WASG4 —> PBITIS).

11. Estimations of relations omitting the first six quarters, which encompass almost allmissing data, are not significantly or materially different from the results reported here.

12. The estimation and predictive-ability tests were repeated alternatively holding out I, 2.or 4 quarters with nearly identical results.

13. No predictive-ability inferences were materially different from the results reportedearlier.

14. At the suggestion of a reviewer, we investigated "distortion" in the company'sperformance targets (Baker 2002); that is. we test whether distributors' performanceratings (red. yellow, green) are consistent with profitability. We regressed quarterlydistributor profitability {PBITIS) on the contemporaneous number of red. yellow, andgreen ratings received on DBSC measures. The results show negative associations withred (p < O.(K)1) and yellow (p = 0.192) ratings and positive associations with greenratings {p = 0.004). These results indicate no performance target distortions that mightexplain lack of observed cause and effect between DBSC measures.

15. This is a major point of'grounded theory" approaches to qualitative research (e.g.,O'Connor. Rice. Peters, and Veryzer 2003; Dougherty 2002; Corbin and Strauss 1990).

16. The logic of the significant, nonlinear, one-period lag effect instead in column 4 is notintuitively obvious.

17. Intrigued by this result, we also estimate the lagged, nonlinear (log) CSAT —> WASGmodel with up to 28 observations and repeated the estimation and predictive-abilitytests. These tests are hampered by the need to omit negative sales growth values, butCSAT4 is significant {p < 0.05). Predictive ability, while better than a constrainedmodel by 1.26 percent, was not significantly improved. Thus, even censored data thatare most favorable to causality refute cause and effect.

18. We gratefully acknowledge a reviewer's constructive comments to improve this figure.



References

Abemeihy. M, M. Home, A. Lillis. M. Malina. and F. Selto. 2005. A mu Iti-methodapproach to building causal performance maps from expert knowledge. Managemenl

Accounting Research 16(2): 135-55.Abemethy, M., and A. Lillis. 2001. Interdependencies in organization design: A test in

hospitals. Journal of Management Accounting Research 13: 107-29.Ambrosini, V., and C. Bowman. 2002. Mapping successful organizational routines. In

Mapping Strategic Knowledge, eds. A. Huff and M. Jenkins, 19-45. Thousand Oaks.

CA: Sage.Anthony, R.. and V. Govindarajan. 1998. Management control. Burr Ridge, IL: Irwin.Arbnor. I., and B. Bjerke. 1997. Methodology for creating business knowledge. London:

Sage.Ashley, R., C. W. J. Granger, and R. Schmalansee. 1980. Advertising and aggregate

consumption: An analysis of causality. Econometrica 48 (5): 1149-67.Baker. G. 2002. Distortion and risk in optimal incentive contracts. Journat of Human

Resources 31(A):12%'S\,Banker, R., G. Potter, and D. Srlnivasan. 2000. An empirical investigation of an incentive

plan that includes nonfinancial performance measures. The Accounting Review 75 (1):65-92.

Bollen, K. 1989. Structural equations with latent variables. New York: Wiley.Bourguignon, A.. V. Malleret, and H. N0rreklit. 2004. The American balanced scorecard

versus ihe French tableau de bord: The ideological dimension. ManagementAccounting Researth 15 (2): 107-34.

Bryant, L., D. Jones, and S. Widener. 2004. Managing value creation within the firm: Anexamination of multiple perfonnance measures. Journal of Management AccountingResearch 16: 107-31.

Bumey, L., C. Henle, and S. Widener. 2006. Do characteristics of strategic pertbrmancemeasurement systems used in incentives enhance organizational fairness? Workingpaper. Rice University.

Camjthers. B. G. 1995. Accounting, ambiguity, and the new institutionalism. Accounting,

Organizations and Society 20 (A): 313-28.Chenhall, R. 2003. Management control systems design within its organizational context:

Findings from contingency-based research and directions for the future. Accounting,Organizations and Society 28 (2-3): 127-68.

Cook, T., and D. Campbell. 1979. Quasi-experimeniation: Design and analysis issues forfield settings. Boston: Haughton Mifflin Company.

Corbin, J., and A. Strauss. 1990 Grounded theory research: Procedures, canons, andevaluative criteria. Qualitative Sociology 13 (I): 3-21.

Darnell, A. 1994. A dictionary of econometrics. Hants, UK: Edward Elgar Publishing.Datar, S., S. Culp, andR. Lambert. 2001. Balancing performance measures./ourna/o/

Accounting Research 39 (I): 75 -94 .

de Geus, A. 1994. Modeling to predict or to leam? In Modeling for Learning Organizations,eds. J. Morecroft and J. Sterman, xiii-xvl. Portland, OR: Productivity Press.

Dearden, J. 1969. The case against ROI control. Harvard Business Review Al (5): 124-35.



d'Iribame, P. 1994. The honour principle in the "bureaucratic phenomenon". OrganizationStudies 15 (I): 81-97,

Dougherty, D. 2002. Grounded theory research methods. In Blackwell Companion to

Organizations. 849-66, Oxford: Blackwell Publishing.Drazin, R., and A. Van de Ven. 1985. Altemative forms of fit in contingency theory.

Administrative Science Quarterly 30 (4): 514-39.Eccles, R. 1991. The performance measurement manifesto. Harvard Business Review

69(1): 131-7.Edwards, P. 1972. The encyclopaedia of philosophy, vols. i - 8 . New York: Macmillian

Publishing Co. and Free Press.Feltham. G. A., and J. Xie. 1994. Performance measure congruity and diversity in multi-task

principal/agent relations. The Accounting Review 69 (3): 429-55.Forrester, J. 1994. Policies, decisions, and information sources for modeling. In Modeling

for Learning Organizations, eds. J. Morecroft and J. Sterman, 51 -84. Portland, OR:Productivity Press.

Friese, S. 1999. Self-concept and identity in a consumer society: Aspects of symbolic

product meaning. Marburg, Germany: Tectum.Frigo, M. 2002a. Nonfinancial perfomiance measures and strategy execution. Strategic

Finance 84 (2): 6-9.Frigo, M. 2002b. Strategy-focused performance measures. Strategic Finance 84 (3): 14-5.Granger, C. W. J. 1969. Investigating causal relations by econometric models and cross-

spectrai methods. Econometrica 37 (3): 424-38.Granger, C. W. J, 1980. Testing for causality; A personal viewpoint. Journal of Economic

Dynamics and Control 2 (4): 329-52.Green, T. 1992. Performance and motivation strategies for today's worlôrce: A guide to

expectancy theory applications. Westport, CT: Greenwood.Huff, A., and M. Jenkins, eds. 2002. Mapping Strategic Knowledge. London: Sage.Ijiri, Y. 1978. The foundations of accounting measurement: A mathematical, economic, and

behavioral inquiry. Houston, TX: Scholars Book Co.Ittner. C , and D. Larcker. 1998. Are non-financial measures leading uidicators of financial

performance? An analysis of customer satisfaction. Journal of Accounting Research36 (Supplement): 1-35.

Ittner, C , and D. Larcker. 2001. Assessing empirical research m managerial accounting: A

value-based management perspective. Journal of Accounting and Economics 32 (1-3):349-410.

Ittner. C , and D. Larcker. 2003. Coming up short on nonfinancial performancemeasuTemenl. Harvard Business Review &\ (11): 88-95.

Ittner, C, D. Larcker, and M. Meyer. 2003. Subjectivity and the weighting of performancemeasures: Evidence from a balanced scorecard. The Accounting Re\'iew 78 (3): 725-58.

Johnston. J. 1994. Econometric methods, 3rd ed. New York: McGraw Hill.Kaplan, R., and D. Norton. 1992. The balanced scorecard — Measures that drive

performance. Hansard Business Review 70 (1): 71 - 9 .Kaplan. R., and D. Norton. 1996. The balanced scorecard: Translating strategy into action.

Boston: Harvard Business School Press.



Kaplan. R., and D. Norton. 200\. The strategy-focused organization. Boston: HarvardBusiness School Press.

Kaplan. R.. and D. Norton. 2005. The balanced scorecard: Measures that drive perfonnance.

Harx'ard Business Review' 83 (7-8): 172-80.Lipe, M., andS. Salterio. 2002. A note on ihejudgmental effects of the balanced scorecard's

information organization. Accounting. Organizations and Society 27 (6): 531 -40.Locke. E.. andG. Latham. 1990. A theory of goal setting and task performance. Englewood

Cliffs. NJ: Prentice Hall.Lufl. J.. and M. Shields. 2002. Leaming the drivers of financial performance: Judgment and

decision effects of financial measures, nonfinancial measures, and statistical models.Working paper, Michigan State University.

Magretia, J. 2002. Why business models matter. Harvard Business Review 80 (5): 86-92.Malina. M.. and F. Selto. 2001. Communicating and controlling strategy: An empirical

study of the effectiveness of the balanced scorecard, Journat of ManagementAccounting Research 13:47-90.

Malina. M.. and F. Selto. 2004. Choice and change of perfonnance model measures.Managemeni Accounting Research 15 (4): 441-60.

Matiessich. R. 1995. Conditional-normative accounting methodology: Incorporating valuejudgments and means-end relations of applied science. Accounting, Organizations and

Society 20 (4): 259-85.Merchani, K. 1998. Modern management control systems. Upper Saddle River, NJ: Prentice

Hall.Meyer. J.. and B. Rowan. 1977. Institutionalized organizations: Formal structure of myth

and ceremony. American Journal of Sociology 83 (2): 340-63.Miller. P., and T O'Leary, 1987. Accounting and the construction of the govemable person.

Accounting, Organizations and Society 12 (3): 235-65.Morecroft. J.. R. Sanchez, and A. Heene. eds. 2002. Systems Perspectives on Resources.

Capabilities, and Management Processes. Amsterdam: Pergamon.Morecroft, J., and J. Sterman. eds. 1994. Modeling for Learning Organizations. Portland.

OR: Productivity Press.Nonaka. I. 1994. A dynamic theory of organizational knowledge creation. Organization

Science 5 ii): 14-38.

Nonaka. I., and H. Takeuchi. 1995. The knowledge-creating company. New York: OxfordUniversity Press.

Norrekli!. H. 2{HX). The balance on the balanced scorecard: A critical analysis of some of itsassumptions. Martagerrient Accounting Research II (1): 65-88.

Nwrreklit, H., L. N0rreklit, and F. Mitchell. 2007. Theoretical conditions for validity in

accounting performance measurement. In Business Performance Measurement —Unifying Theory and Integrating Practice, ed. A. Neely. Cambridge: CambridgeUniversity Press.

Norreklir. L. 1987. Format .structures in social logic. Aalborg. DK: Aalborg UniversityPress,

N0rreklit. L.. H. Neneklit, and P. Israelsen. 2006. Validity of management control topoi:Towards constructivisi pragmatism. Mumigemeru Accounting Research 17(1): 42-71.

. 24 No. 3 (Fall 2007)


O'Connor. C , M. Rice, L. Peters, and R. Veryzer. 2003. Managing interdisciplinary,

longitudinal research teams: Extending grounded theory-building methodologies.Organization Science H (4): 353-73.

Popper, K. 1959. The logic of scientific discovery. Translation of Logik der Forschung.

London: Hutchinson.Popper, K. 1961. The poverty of historicism, 2nd ed. London: Routledge.Popper, K. 1963. Conjectures and refutations: The growth of scientific knowledge. London:

Routledge.

Porter, M. 1985. Competitive advantage. New York: Free Press.Ridgway. V. F. 1956. Dysfunctional consequences of performance measurements.

Administrative Science Quarterly 1 (2): 240-7,Rucci, A., S. Kim, and R. Quinn. 1998. The employee-customer-profit chain at Sears.

Harvard Business Revien- 76 (1): 82-97.Sanchez, R., A. Heene, and H. Thomas, eds. 1996. Dynamics of Competence-Based

Competition. Oxford, UK: Pergamon.Searle, J. R. 1995. The construction of social reality. New York: Free Press.Shields. M.. and S. M. Young. 1989. A behavioral model for implementing cost

management systems. Journal of Cost Management 2(1): 29-34.Simons, R. 2000. Performance measurement and control systems for implementing strategy.

Upper Saddle River, NJ: Prentice Hall.Slife, B. D.. and R. N. Williams. 1995. What's behind the research? Discovering hidden

assumptions in the behavioral sciences. London: Sage.Willard, B. 2005. The NEXTsusiainabitity wave: Building boardroom buy-in. Gabriola

Island, BC: New Society Publishers.Zimmerman, J. 1997. Accounting for decision making and control. Burr Ridge, IL: Irwin-

McGraw-Hill.


Documents

Relations Among Measures Climate of Control and PM Models