SSRN-id513260

8/18/2019 SSRN-id513260

1/26

Accounting and Finance 44 (2004) 1–26

© AFAANZ, 2004. Published by Blackwell Publishing.

Impact of intelligent decision aids on expert

and novice decision-makers’ judgments

Vicky Arnolda, Philip A. Collier b,

Stewart A. Leech

b

, Steve G. Sutton

a,b

a

University of Connecticut, Storrs, CT 06269, USA

b

Department of Accounting and Business Information Systems, Faculty of Economics and

Commerce, The University of Melbourne, Melbourne, 3010, Australia

Abstract

Businesses have invested tremendous resources into intelligent decision aiddevelopment. A good match between user and aid may improve the expert

decision-maker’s decision quality. However, novices may be prone to poorer

decision-making if intelligent decision aids are more expert than the user.

The present paper provides an empirical test of the impact of decision aids on

subjects with differential expertise levels. The results support the contention

that intelligent decision aids aggravate bias in novices’ decision-making but

mitigate bias in experts’ decision-making processes. Intelligent decision aids

may be best viewed as complements to expert decision-makers during complex

problem analysis and resolution.

Key words

: Expert systems; Intelligent decision aids; Technology dominance;

Insolvency; Judgement/Decision making

JEL classification

: M41

We

gratefully acknowledge financial support from the Australian Research Council, the Insti-tute of Chartered Accountants in Australia (IT Chapter), and the University of Massachusetts

Healy Foundation; and participation by the following organisations: Arthur Andersen,Brooke Bird, Commonwealth Bank of Australia, Paul J. Cook and Associates, Court & Co.,Deloitte Touche Tohmatsu, Ernst & Young, Ferrier Hodgson, Grant Thornton, Jones Condon& Co., KPMG, National Australia Bank Limited, PricewaterhouseCoopers, Sims Lockwood& Partners, and Wise Lord & Ferguson. We are also grateful for the computer programmingsupport provided by Nicole Clark. The present paper has benefited from comments received byparticipants and discussants at the Accounting Information Systems Research Symposium,American Accounting Association (AAA) Auditing Section Midyear meeting, InternationalSymposium on Audit Research, AAA Annual meeting and workshops at the University of Melbourne, Michigan State University, Texas Tech University, University of Central Florida,and the University of Connecticut.

Received 20 December 2002; accepted 30 January 2003 by Robert Faff (Editor).

8/18/2019 SSRN-id513260

2/26

2 V. Arnold et al.

/ Accounting and Finance 44 (2004) 1–26

© AFAANZ, 2004

1. Introduction

Nagle (1999) noted that one of the biggest issues faced by knowledge-driven

organisations is how to capture, share and retain the knowledge of an organisa-

tion’s professionals – particularly when there is a high risk of key individuals

electing to leave the organisation. One of the methods that such organisations

have frequently used to facilitate knowledge management over the past two

decades has been the development and deployment of intelligent decision aids.

Intelligent decision aids, also referred to as expert systems or intelligent sys-

tems, are software-intensive systems that integrate the expertise of one or more

experts in a given decision domain. Such aids are intended to provide a specific

recommendation to a given problem and/or provide expert advice that assists

the user in making a better decision than when unaided.

The major impetus for the initial rush to develop intelligent decision aids

was the perceived benefits of reduced labour costs and increased sharing of

firm expertise (e.g., Elliott and Jacobson, 1987; Willingham and Ribar, 1987).

Despite the broad acclaim and heavy push for intelligent decision aids that

could enable novices to make decisions they were not individually prepared

to perform, these systems generally fell short of the desired improvements

in decision-making efficiency. Indeed, the very system Willingham and Ribar

(1987) were publicising, KPMG’s LoanProbe, had a very short life. It was soon

recognised that junior level staff were incapable of providing the basic level of

judgement needed to correctly enter the information required by LoanProbe.

Thus, the intelligent decision aid was rendered incapable of producing accuratedecisions from the inaccurate information provided to the system. In effect, the

users were unable to provide the rudimentary level of expertise needed to use

the system.

Rochlin (1997; p. 167) notes that the failure of intelligent systems in complex,

high technology environments has been systemic (e.g., Perrow, 1984; Rochlin

et al.

, 1987; Demchak, 1991; La Porte and Consolini, 1991; Roberts, 1993;

Sagan, 1993). Several researchers have questioned why such systems have not

had better success (e.g., Duchessi and O’Keefe, 1992; Bouwman, 1996). The lack

of desired success has been associated with nonacceptance of the technology, iden-

tification of inappropriate problem domains, a general lack of perceived benefits,effort minimisation by users, or potential legal liability risks (Abdolmohammadi,

1999; Duchessi and O’Keefe, 1992; Sutton et al.

, 1995; Todd and Benbasat, 1999).

Bouwman (1996) notes, however, that these reasons are simply symptomatic

of problems and further investigation is needed to identify the underlying

causes of such problems and the factors that lead to implementation success or

failure.

Much work has been completed in the academic environment related to

the factors determining usage of and reliance on deterministic decision aids.

These decision aids, however, are less orientated towards complex judgement

tasks with subjective solutions. Rather, the aids used to facilitate much of this

https://www.researchgate.net/publication/288254021_Audit_technology_A_heritage_and_a_promise?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284495971_Development_of_an_expert_audit_system_for_loan_loss_evaluation?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/245232055_Normal_Accidents_Living_With_High_Risk_Technologies?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/272934251_Military_Organizations_Complex_Machines_Modernization_in_the_US_Armed_Forces?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284422324_Working_in_theory_but_not_in_practice_Theoretical_challenges_in_high_reliability_organizations?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/274450886_The_Limits_of_Safety_Organizations_Accidents_and_Nuclear_Weapons?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/290117789_A_comprehensive_taxonomy_of_audit_task_structure_professional_rank_and_decision_aids_for_behavioral_research?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/290117789_A_comprehensive_taxonomy_of_audit_task_structure_professional_rank_and_decision_aids_for_behavioral_research?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4941659_Contrasting_successful_and_unsuccessful_Expert_Systems?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/271898608_An_Analysis_of_Potential_Legal_Liability_Incurred_Through_Audit_Expert_Systems?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/271898608_An_Analysis_of_Potential_Legal_Liability_Incurred_Through_Audit_Expert_Systems?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/271898608_An_Analysis_of_Potential_Legal_Liability_Incurred_Through_Audit_Expert_Systems?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079922_Evaluating_the_Impact_of_DSS_Cognitive_Effort_and_Incentives_on_Strategy_Selection?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/290117789_A_comprehensive_taxonomy_of_audit_task_structure_professional_rank_and_decision_aids_for_behavioral_research?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/290117789_A_comprehensive_taxonomy_of_audit_task_structure_professional_rank_and_decision_aids_for_behavioral_research?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/274450886_The_Limits_of_Safety_Organizations_Accidents_and_Nuclear_Weapons?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284422324_Working_in_theory_but_not_in_practice_Theoretical_challenges_in_high_reliability_organizations?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284495971_Development_of_an_expert_audit_system_for_loan_loss_evaluation?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/288254021_Audit_technology_A_heritage_and_a_promise?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/271898608_An_Analysis_of_Potential_Legal_Liability_Incurred_Through_Audit_Expert_Systems?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/272934251_Military_Organizations_Complex_Machines_Modernization_in_the_US_Armed_Forces?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/245232055_Normal_Accidents_Living_With_High_Risk_Technologies?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079922_Evaluating_the_Impact_of_DSS_Cognitive_Effort_and_Incentives_on_Strategy_Selection?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4941659_Contrasting_successful_and_unsuccessful_Expert_Systems?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

3/26

V. Arnold et al.

/ Accounting and Finance 44 (2004) 1–26 3

© AFAANZ, 2004

research have generally been designed to assist in solving difficult mechanical

aggregation tasks to attain objective solutions. An array of factors that limit

adoption and use have been identified – including experience with the aid (Taylor

and Todd, 1995; Whitecotton, 1996), confidence in one’s own decision making

ability (Mackay and Elam, 1992; Davis and Kottemann, 1994; Whitecotton,

1996), and reluctance to render control of the decision process to a computerised

aid (Arkes et al.

, 1986; Kleinmuntz, 1990; Boatsman et al.

, 1997). The general

consensus of the research has been that users make suboptimal decisions and

are often outperformed by a decision aid.

These results may not be applicable, however, in situations where a judgement-

based intelligent decision aid, expert decision makers, and/or collaborative

designs for decision aids are involved. First, Whitecotton et al.

(1998) note that

most decision aid studies use deterministic aids, and the user is not permitted

to have access to and use additional information beyond that fed into the deci-

sion aid. They suggest that the results will likely be very different when dealing

with non-deterministic problems and when allowing the user access to additional

information. Second, Mackay and Elam (1992) demonstrate that experts and

novices use decision aids very differently and accordingly perform very differ-

ently – even when using a relatively simple aid. Experience with using the aid

was also shown to be critical to performance. Third, work in collaborative systems

design provides support for the belief that where interactive decision aids

and the user trade off control of the decision process, better results are gained

than either the aid or user could achieve individually (Hale and Kasper, 1989;

Kasper, 1996).Given the potential differences that may exist in decision aid effects under

the three noted conditions, two experiments were designed to explore these

factors. First, a complex intelligent decision aid that incorporates knowledge

acquired from multiple expert decision-makers in the area of corporate insolv-

ency was utilised. The intelligent decision aid was designed to incorporate the

basic concepts of collaborative design. Second, the theory of technology dom-

inance (Arnold and Sutton, 1998) was applied as the basis for formulating

hypotheses differentiating between expected decreased performance by novice

decision makers and expected increased performance by expert decision

makers. To test the hypotheses, 167 insolvency practitioners (corporate recoveryspecialists) from several major firms, representing all experience levels, were

used as participants. Third, a highly unstructured corporate insolvency task was

used in order to match the task with the high-level intelligent decision aid and

to challenge the participants. Since no single right answer exists for such an

unstructured task, the amount of information processing bias incorporated into

the decision makers’ judgements was used as a surrogate for overall decision

performance. The results show a significant increase in the bias exhibited by

novices using the intelligent decision aid (i.e., poorer judgement) while the bias

was reduced for experts when using the intelligent decision aid (i.e., improved

judgement).

https://www.researchgate.net/publication/242637310_Assessing_IT_Usage_The_Role_of_Prior_Experience?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/242637310_Assessing_IT_Usage_The_Role_of_Prior_Experience?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/286336800_The_effects_of_experience_and_confidence_on_decision_aid_reliance_A_causal_model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079491_A_Comparative_Study_of_How_Experts_and_Novices_Use_a_Decision_Aid_to_Solve_Problems_in_Complex_Knowledge_Domains?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/286336800_The_effects_of_experience_and_confidence_on_decision_aid_reliance_A_causal_model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/286336800_The_effects_of_experience_and_confidence_on_decision_aid_reliance_A_causal_model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/20966311_Why_we_still_use_our_heads_instead_of_the_formulas_Toward_an_integrative_approach_Psychological_Bulletin_107_296-310?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079672_A_Theory_of_Decision_Support_System_Design_for_User_Calibration?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/286336800_The_effects_of_experience_and_confidence_on_decision_aid_reliance_A_causal_model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/286336800_The_effects_of_experience_and_confidence_on_decision_aid_reliance_A_causal_model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/286336800_The_effects_of_experience_and_confidence_on_decision_aid_reliance_A_causal_model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/20966311_Why_we_still_use_our_heads_instead_of_the_formulas_Toward_an_integrative_approach_Psychological_Bulletin_107_296-310?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/4819499_The_Effects_of_Decision_Consequences_on_Auditors'_Reliance_on_Decision_Aids_in_Audit_Planning?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/242637310_Assessing_IT_Usage_The_Role_of_Prior_Experience?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/242637310_Assessing_IT_Usage_The_Role_of_Prior_Experience?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079672_A_Theory_of_Decision_Support_System_Design_for_User_Calibration?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079491_A_Comparative_Study_of_How_Experts_and_Novices_Use_a_Decision_Aid_to_Solve_Problems_in_Complex_Knowledge_Domains?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

4/26

4 V. Arnold et al.


© AFAANZ, 2004

The remainder of the present paper is divided into four sections. In Section

2, the published literature on decision aids is explored further and the theory of

technology dominance is introduced. The research method is explained in Sec-

tion 3, and the results of the empirical tests are discussed in Section 4. Section

5 provides an overview of the results and details the implications for intelligent

decision aid design and use in complex decision environments.

2. Theory development and research hypotheses

Arnold and Sutton (1998) presented the theory of technology dominance as a

model for understanding the conditions under which use of an intelligent deci-

sion aid are more likely to lead to success or failure. The theory focuses on the

factors impacting the use of a well-designed intelligent decision aid, not the

design of such aids. Arnold and Sutton (1998) note that a basic requirement of suc-

cess for an intelligent decision aid is the achievement of some level of reliance

on the aid by a user.

2.1. Decision aid reliance

Reliance implies two conditions – acceptance and influence. Acceptance

means that the user adopts the aid as a useful part of the decision-making pro-

cess. Influence means that once the user adopts the aid as a useful part of the

process, the user not only enters data and receives responses, but allows the

process of entering data and receiving responses to become part of the user’s judgement formulation. Thus, while some researchers have implied reliance is

adherence to the decision aid’s recommendations (e.g., Brown and Jones,

1998), the theory of technology dominance adopts the broader definition of reli-

ance that the decision aid becomes a part of the decision-making process and

exerts some influence on decision outcomes.

In determining the factors leading to reliance, Arnold and Sutton (1998)

synthesised the findings of prior studies. In every prior study in which novice

subjects were used, reliance occurred (see Table 1). They note that this is

intuitively logical since most professionals pursue help in completing a task

when they lack the ability to perform the task on their own; hence, inexperi-enced subjects would be expected to rely on the decision aid. This finding is

consistent with the work of Todd and Benbasat (1991, 1992, 1994, 1999, 2000)

which suggests that decision aid users will minimise work effort when faced

with a decision making task.

If higher levels of task experience are present, then reliance is less assured.

Rather, Arnold and Sutton (1998) note that the results of studies using moderate

to highly experienced subjects are inconsistent (see Table 2). However, three

different factors were identified as critical to reliance for experienced subjects:

decision aid familiarity; task complexity; and cognitive fit. Decision aid fam-

iliarity relates to whether the user is familiar with the aid and has previously

https://www.researchgate.net/publication/228301403_Factors_that_Influence_Reliance_on_Decision_Aids_A_Model_and_an_Experiment?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228301403_Factors_that_Influence_Reliance_on_Decision_Aids_A_Model_and_an_Experiment?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079904_An_Experimental_Investigation_of_the_Impact_of_Computer_Based_Decision_Aids_on_Decision_Making_Strategies?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079904_An_Experimental_Investigation_of_the_Impact_of_Computer_Based_Decision_Aids_on_Decision_Making_Strategies?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228301403_Factors_that_Influence_Reliance_on_Decision_Aids_A_Model_and_an_Experiment?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228301403_Factors_that_Influence_Reliance_on_Decision_Aids_A_Model_and_an_Experiment?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

5/26

© AF AA NZ

,2 0 0 4

Table 1

Relationship between low levels of task experience and reliance on decision aida

Study Subjects Task Decision aid

Eining et al. (1997) 116 auditors Predict likelihood of

management fraud

Checklist

Logit model

Expert system

Whitecotton (1996) 35 graduate students Forecast earnings Statistical model

Ghosh & Whitecotton (1987) 30 graduate students Forecast earnings Statistical modelGlover et al. (1996) 90 undergraduate students Determine tax liability Computerised worksheet

Brown and Jones (1998) 116 undergraduate students Recommend a new

computer system

Recommended solution

Ashton (1992) 59 auditors Predict bond rating Statistical model

Shome and Ibrahim (1997) 80 students in CA training Likelihood of going

concern

Checklist

Davis and Kottemann (1994) 52 MBA students Plan level of production Decision support system

(what if analysis)

CA, Chartered Accountants.a Based on the analysis presented by Arnold and Sutton (1998).

8/18/2019 SSRN-id513260

6/26

© AF AA NZ ,2

0 0 4

Table 2

Factors affecting decision aid reliance for experienced decision-makersa

Study Subjects Task Decision aid Task experience

Decision aid

familiarity

Davis (1998) 206 auditors Likelihood of

going concern

Statistical model Moderate to high No

Checklist Moderate to high Yes

Kachelmeier and

Messier (1990)

152 auditors Predict sample size

for non-statistical

sample

Computational aid Moderate Yes

Bonner et al. (1996) 105 auditors Assess probabilities

of errors

List aid Moderate Yes

Statistical model Moderate ND

b

List aid and

statistical model

Moderate NDb

Whitecotton (1996) 40 financial analysts Forecast earnings Statistical model High Yes

Ghosh & Whitecotton,

(1987)

24 financial analysts Forecast earnings Statistical model High Yes

Boatsman et al. (1997) 118 auditors Predict management

fraud

Completed

checklist

Moderate No

a Based on the analysis presented by Arnold and Sutton (1998).b Little information regarding explanation and presentation of the mechanical aggregation aid (statistical model) was provi

therefore, judging whether the information was similar to the type the decision-makers would have available to use in th

possible.

8/18/2019 SSRN-id513260

7/26

V. Arnold et al.


© AFAANZ, 2004

used it. Mackay and Elam (1992) used a protocol analysis approach to observe

inexperienced and experienced users of a decision aid and noted that usage pat-

terns were positively related to decision aid familiarity. Taylor and Todd (1995)

also show that familiarity with a system shifts the users’ focus from ease of use

and usefulness concerns to control of decision aid processes. Task complexity

is a relative concept; if the task requires greater effort to resolve than the effort

required to use the decision aid, then it is perceived to be of moderate to high

complexity and vice versa. The extensive work by Todd and Benbasat (1991,

1992, 1994, 1999, 2000) on effort aversion provides a level of support for the

task complexity component of the model. Chau and Hu (2001) provide even

more evidence as they studied medical professionals and found such decision

makers were only willing to adopt technology that would facilitate solution of

problems that could not be more easily completed without the aid. Cognitive fit

includes Vessey’s (1991) notion of correspondence between presentation mode

and task. In the theory of technology dominance, cognitive fit also includes the

‘congruence in the cognitive decision processes and prompted cognitive reason-

ing between the aid and the user’ (Arnold and Sutton, 1998, p. 180). Mathieson

and Keil (1998) study system users’ decision processes in selecting to use a system

and found significant evidence supporting task technology fit (based on cognitive fit

of the technology) as key to acceptance. The theory of technology dominance

posits that to enhance the likelihood of reliance on a decision aid by an experi-

enced decision maker: (i) the task should be highly complex; (ii) the decision

aid should be familiar; and (3) the cognitive fit between aid and user should be

congruent.

2.2. Technology dominance

Technology dominance is the state of decision-making whereby the decision

aid, rather than the user, takes primary control of the decision-making process.

Rochlin (1997) notes that the user should be able to step in and take over, but

rarely is the inexperienced user adequately prepared to do so. Rochlin suggests

the real risk is that a user with limited expertise is unable to properly use (or

might misinterpret) the output of an intelligent decision aid. The key is that

input judgements and output decisions require some sophistication on the partof the user in order to be adequately interpreted. Thus, even if the intelligent

decision aid functions properly, the end decision can still result in failure. One

example of such a problem is the Iranian commercial jet that was shot down

during the U.S. embargo of 1988. The sophisticated air defense systems worked

perfectly, but the lower level officer monitoring and reporting the intelligent

decision aid’s output misinterpreted the results and the jet was misidentified as

a military aircraft (Rochlin, 1997, p. 165).

The problems may not only occur from failure to interpret system outputs but

also by contamination of outputs through inaccurate input of supporting data.

Mackay and Elam (1992) observed novice decision makers use of a decision aid

https://www.researchgate.net/publication/220079904_An_Experimental_Investigation_of_the_Impact_of_Computer_Based_Decision_Aids_on_Decision_Making_Strategies?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/227770329_Information_Technology_Acceptance_by_Individual_Professionals_A_Model_Comparison_Approach?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079491_A_Comparative_Study_of_How_Experts_and_Novices_Use_a_Decision_Aid_to_Solve_Problems_in_Complex_Knowledge_Domains?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079904_An_Experimental_Investigation_of_the_Impact_of_Computer_Based_Decision_Aids_on_Decision_Making_Strategies?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/227770329_Information_Technology_Acceptance_by_Individual_Professionals_A_Model_Comparison_Approach?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079491_A_Comparative_Study_of_How_Experts_and_Novices_Use_a_Decision_Aid_to_Solve_Problems_in_Complex_Knowledge_Domains?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

8/26

8 V. Arnold et al.


© AFAANZ, 2004

and found that the users repetitively viewed surface level decision support

models, but failed to access or supply data necessary for deep level decision

support models.

Arnold and Sutton (1998) note that such technology dominance is unlikely to

occur based on the use of a checklist or mechanical aggregation aid – the types

of aids most prevalent in the decision support systems research literature.

Rather, they note that the concerns over technology dominance emanate from

decision domains that are highly subjective in nature – the type of domains

where intelligent decision aids would predominate. It is these systems that are

of interest in the theory and in the present study.

Hence, the concerns over the use of intelligent decision aids are that the user

will lack adequate expertise to make a decision without the aid, the aid will be

perceived as capable of making the decision, and the user will become depend-

ent on the aid for completing the task. Todd and Benbasat (2000) found in their

study that such users readily accepted available normative models when they

were easily accessed. The mismatch (in terms of expertise) between the user

and the intelligent decision aid makes the novice user susceptible to dominance

by the technology. Arnold and Sutton (1998) posit that this dominance may be

increased by the users’ own sense of control which emanates from interaction

with the aid. Davis and Kottemann, 1994) found evidence to support the theory

of illusion of control (Langer, 1975), which proposes that user control over

inputs into the system may create an illusion of control over the decision aid

and result in overreliance on the outputs. Kahai et al.

(1998) found that the illu-

sion of control resulted in lower preferences for thinking over a decision beforemaking a final judgement.

Evidence from multiple domains regarding misuse of intelligent decision aids

and decision failure, coupled with related conceptual concerns surrounding the

mismatch in expertise between aid and decision-maker, led Arnold and Sutton

(1998) to posit that the likelihood of poor decision-making increases when novices

use an intelligent decision aid. Their stated proposition is as follows:

When the expertise of the user and intelligent decision aid are mismatched, there is a

negative relationship between the user’s expertise level and the risk of poor decision-

making. p. 185

Essentially, Arnold and Sutton (1998) take the position that if the aid has

greater decision-making capability than the user, the user may become subser-

vient to the intelligent decision aid and therefore subject to technology domin-

ance. Studies of the use of tax preparation software by novice tax preparers

provides evidence of such technology dominance as novices overreacted to

audit flags in the software and over reported tax liability (Masselli et al.

, 2002)

and novice tax preparers taught through tax preparation software were unable

to prepare tax returns manually (Noga and Arnold, 2002). This latter case rein-

forces the concern that poor decision-making becomes even more likely if the

https://www.researchgate.net/publication/240126083_Inducing_Compensatory_Information_Processing_Through_Decision_Aids_that_Facilitate_Effort_Reduction_An_Experimental_Assessment?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/240263547_User_Perceptions_of_Decision_Support_Effectiveness_Two_Production_Planning_Experiments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/230726576_The_Illusion_of_Control?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/238806572_Active_involvement_familiarity_framing_and_the_illusion_of_control_during_decision_support_system_use?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/238806572_Active_involvement_familiarity_framing_and_the_illusion_of_control_during_decision_support_system_use?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/238806572_Active_involvement_familiarity_framing_and_the_illusion_of_control_during_decision_support_system_use?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/281160404_The_impact_of_embedded_intelligent_agents_on_tax_compliance_decisions?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/281160404_The_impact_of_embedded_intelligent_agents_on_tax_compliance_decisions?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/281160404_The_impact_of_embedded_intelligent_agents_on_tax_compliance_decisions?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/247111792_Do_tax_decision_support_systems_affect_the_accuracy_of_tax_compliance_decisions?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/281160404_The_impact_of_embedded_intelligent_agents_on_tax_compliance_decisions?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/238806572_Active_involvement_familiarity_framing_and_the_illusion_of_control_during_decision_support_system_use?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/240263547_User_Perceptions_of_Decision_Support_Effectiveness_Two_Production_Planning_Experiments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/230726576_The_Illusion_of_Control?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/240126083_Inducing_Compensatory_Information_Processing_Through_Decision_Aids_that_Facilitate_Effort_Reduction_An_Experimental_Assessment?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/247111792_Do_tax_decision_support_systems_affect_the_accuracy_of_tax_compliance_decisions?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

9/26

V. Arnold et al.


© AFAANZ, 2004

intelligent decision aid fails and the decision-maker is left facing a decision not

encountered before (Gal and Steinbart, 1987).

This technology dominance argument does not imply that intelligent decision

aids should be abandoned. Rather, the theory posits that success will come from

an aid that is well matched with a user. The US military is demonstrative of this

matching between user and aid. Demchak (1991) notes that what started as a

philosophy known as Smart Machine – Dumb Maintainer did not work. The US

military quickly learned that smart machines necessitate smart people to operate

them. Instead of trained labour becoming expendable, retaining the experts

capable of using the system became a high priority (Rochlin, 1997).

Arnold and Sutton (1998) couch their arguments in terms of the desirability

of an electronic colleague in cases where there is a good match between the aid

and the decision-maker’s expertise. The electronic colleague becomes a partner

in the decision process, transforming individual decision-making into a dyadic

group decision-making mode. Rather than a user making a decision using a

tool, the decision environment becomes a user making a decision after consulting

a colleague – albeit electronic. Mackay and Elam (1992) provide some support

for this concept as their observation of experienced decision makers using a

simple decision aid found that experienced decision makers completed much

deeper level analysis of alternative models when a familiar decision aid was

available than did equivalent decision makers without an aid.

The work in collaborative systems design provides support for the belief

that where interactive intelligent decision aids and the user balance control of

the decision process, results are better than either the aid or user could achieveindividually (Hale and Kasper, 1989; Kasper, 1996). Eining et al.

(1997) focused

on the development of a constructive dialogue as the interface to their intelli-

gent decision aid. The idea behind the constructive dialogue was to engage

the user in the decision-making process of the intelligent decision aid. Eining

et al.

appeared to be successful in engaging users in an interactive decision

process through such an aid, even though the system used was a deterministic

aid.

Ensuring that systems are designed to be collaborative promotes the desired

user/computer interaction. The goal is to encourage the user to adopt the system

as an electronic colleague. If this goal can be achieved, the resulting dyadicdecision-making approach should result in enhanced judgement performance –

similar to two experts working jointly to solve a problem (Arnold and Sutton,

1998). The symbiotic relationship avoids dominance by user or system. The

proposition is stated as follows:

When the expertise level of the user and intelligent decision aid are matched, there is a

positive relationship between reliance on the aid and improved decision-making. p. 187

The difficulty in testing these two propositions is the ambiguity in defining a

better or worse decision in domains that are highly subjective. Thus, one of the

https://www.researchgate.net/publication/288937445_Artificial_intelligence_and_research_in_accounting_information_systems_Opportunities_and_issues?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/272934251_Military_Organizations_Complex_Machines_Modernization_in_the_US_Armed_Forces?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079672_A_Theory_of_Decision_Support_System_Design_for_User_Calibration?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/284107668_The_theory_of_technology_dominance_Understanding_the_impact_of_intelligent_decision_aids_on_decision_makers'_judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/288937445_Artificial_intelligence_and_research_in_accounting_information_systems_Opportunities_and_issues?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/272934251_Military_Organizations_Complex_Machines_Modernization_in_the_US_Armed_Forces?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220079672_A_Theory_of_Decision_Support_System_Design_for_User_Calibration?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

10/26

10 V. Arnold et al.


© AFAANZ, 2004

challenges in the present study is to place the decision environment into an

observable and measurable context, while maintaining the richness of the judge-

ment environment.

2.3. Testable hypotheses for technology dominance

A basic motivation for judgement and decision-making research has been to

support the design of decision aids aimed at overcoming specific memory-

related deficiencies in the judgement process (e.g., Libby, 1985; Silver, 1988;

Bonner et al.

, 1996). The development of such aids should be based on the re-

cognition of the specific sources of judgement error and incorporate research

into the design of aids that target those sources of error. Yet, as Rose (2002, p. 116)

notes, ‘Research has also demonstrated that decision aids can create biases and

intensify shortcomings in human information processing’. The present study

examines such effects through the potential increase in information processing

biases by professional decision-makers. If the theory of technology dominance

holds, novice decision-makers who use an intelligent decision aid would be

expected to have difficulty in balancing conflicting pieces of information, inter-

preting queries for data put forth by an aid, and interpreting an elaborately

discussed recommendation. In the case of conflicting information, the user

may opt to simply replace the first entered information with a second piece of

conflicting information – leading to a loss of the initial information to the aid

in its decision processing. The user may also provide incorrect information in

response to a request from the intelligent decision aid, especially if the user isunable to properly interpret the request. The primary effect is clearly expected

to come from the system’s output, however. The recommendation and explana-

tion provided as output details reasons for the aid’s decision, and reports the

effect of any missing information. The novice user will likely have difficulty

assessing the limitations caused by the missing information, will be unwilling

to exert significant effort to increase understanding in order to reason through

processes, will be overly convinced by the system’s explanation of its decision

(i.e., the report will seem quite expert and convincing to the novice), and will

tend to gravitate toward the decision rendered rather than toward assessment of

the details provided by the system in forming its decision.If such an effect occurs, then novices’ bias should be accentuated. This leads

to the following potential business decision risk based on the first proposition

of the theory of technology dominance noted earlier in this section:

Business decision risk: Novice decision makers using a complex intelligent

decision aid will make significantly worse decisions when using the aid due to

the presence of information processing biases.

If the noted business decision risk comes to fruition, two phenomena should

be observable:

https://www.researchgate.net/publication/275882721_Availability_and_the_Generation_of_Hypotheses_in_Analytical_Review?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/238836684_Descriptive_Analysis_for_Computer-Based_Decision_Support_Special_Focus_Article?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228178500_Using_Decision_Aids_to_Improve_Auditors'_Conditional_Probability_Judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228178500_Using_Decision_Aids_to_Improve_Auditors'_Conditional_Probability_Judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228178500_Using_Decision_Aids_to_Improve_Auditors'_Conditional_Probability_Judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/228178500_Using_Decision_Aids_to_Improve_Auditors'_Conditional_Probability_Judgments?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/238836684_Descriptive_Analysis_for_Computer-Based_Decision_Support_Special_Focus_Article?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/275882721_Availability_and_the_Generation_of_Hypotheses_in_Analytical_Review?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

11/26

V. Arnold et al.


© AFAANZ, 2004

H1: Information processing bias will be present in the novices’ decisions when

aided by an intelligent decision aid.

H2: The amount of information processing bias will be greater when novices

are aided by an intelligent decision aid.

The first hypothesis recognises that the business decision risk is unimportant if

there is not significant bias in the end decisions of novices using an intelligent

decision aid, while the second hypothesis addresses the issue of technology

dominance by examining the incremental impact of using the aid.

Of course, the intent of developing intelligent decision aids was never to

increase business decision risk through increased bias, but rather to improve

decision-making and, in some cases, remove decision bias. Todd and Benbasat

(1999) note that decision aid design is generally based on one of two schools of

thought: (i) decision aid as technological intervention should assist in the

implementation of normative decision-making strategies; or (ii) decision aid as

a behavioural approach with the aim of extending the capabilities and over-

coming the limitations of decision-makers. The first approach has been largely

considered unsuccessful as users tend not to want to give up their own judge-

ment in favour of an aid, albeit due to an inherent desire to circumvent the aid,

personal confidence in judgement capability, lack of involvement with model

development and/or decision consequences (Arkes et al.

, 1986; Boatsman et al.

,

1997; Whitecotton and Butler, 1998; Rose, 2002).

As noted earlier, a primary goal of decision aid research and design has been

more aligned with the latter approach – extending the capabilities and overcom-ing the limitations of decision-makers. A primary target area for such systems

has been prescribed as aids for debiasing. Information processing biases may be

counteracted by an aid that focuses on the sources of such judgement bias. The

theory of technology dominance would posit that such efforts would be more

successful with expert decision makers who understand how to weigh conflict-

ing information and will be able to readily interpret the intelligent decision

aid’s queries. Experts are expected to be less interested in the decision of the

decision aid (for all the reasons noted in the prior paragraph) and more

interested in the reasoning used by the intelligent decision aid to formulate

the decision. Examination of the reasoning process should cause the user torecall key cues that may have faded during processing of large volumes of

data. This recall process helps the expert user focus on the key pieces of

information.

The concern over debiasing through decision aids is based on the following

business decision risk:

Business decision risk: An expert processing large numbers of complex informa-

tion cues will exhibit information processing bias in arriving at end decisions.

The business risk can be restated in hypothesis form as:

https://www.researchgate.net/publication/292725342_Influencing_decision_aid_reliance_through_involvement_in_information_choice?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/292725342_Influencing_decision_aid_reliance_through_involvement_in_information_choice?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

12/26

12 V. Arnold et al.


© AFAANZ, 2004

H3: Information processing bias will be present in the experts’ decisions when

not aided by an intelligent decision aid.

The goal of an intelligent decision aid would be to mitigate such a business

decision risk and can be stated as:

Mitigation of business decision risk: An intelligent decision aid is desirable if

it can reduce the decision risk of an expert processing a large number of com-

plex information cues by reducing information processing bias in the decision

outcome.

Thus, the following hypothesis is used to test whether the intelligent decision

aid in the present study is able to reduce the information processing bias of

experts:

H4: The amount of information processing bias will be reduced when experts

are aided by an intelligent decision aid.

The diminution of information processing bias in experts’ judgements is

predicted based on the use of a collaborative intelligent decision aid. When

the aid is used in this manner, the expert is more likely to recall important cues

discovered earlier in the process as part of the reasoning given for the aid’s

recommendations. This should occur primarily because the expert will be less

interested (and less influenced by) the decision rendered by the intelligent deci-sion aid, but instead more focused on the reasoning provided in the decision

aid’s explanation of its decision. The intelligent decision aid should also help

the expert to overcome the information load associated with combining large

numbers of cues.

3. Research method

Two sets of experiments were conducted to test the four hypotheses. An

insolvency case was used in both sets of experiments with a total of 80 (43

novices and 37 experts) insolvency practitioners having access to an intelligentdecision aid and with 87 (39 novices and 48 experts) practitioners who did not.

The method used in all experimental sessions is discussed in detail in the follow-

ing subsections.

3.1. Decision task

The decision task was set in the domain of insolvency due to its highly spe-

cialised nature. Insolvency practice has existed internationally for most of this

century and can involve Chartered Accountants (CA) taking over management

of a company that is in financial distress and evaluating the best alternative for

8/18/2019 SSRN-id513260

13/26

V. Arnold et al.


© AFAANZ, 2004

the future of that company. In many cases this requires immediate liquidation of the

company. In other cases, the CA may decide that the best return for creditors

and other stakeholders can be obtained by keeping the company in business

(i.e., trading on) for a period of time until the business can be sold as a going

concern or turned around and control returned to the company’s directors.

In developing the case materials, a specific information processing bias was

selected and conditions providing the highest likelihood of susceptibility by

decision makers to the bias were identified for use in the case materials. Hogarth

and Einhorn (1992; p. 1) put forth the belief-adjustment model as ‘a theory of

belief updating that explicitly accounts for order-effect phenomena as arising

from the interaction of information processing strategies and task character-

istics.’ In short, the model predicts that individuals’ judgements will be influ-

enced by the order in which information is received. The order effects posited

in the belief-adjustment model are dependent on task complexity and the nature

of information sequences. In the case of high complexity, the particular order

effect that would be expected is recency – meaning that the last information

received is weighted greater during information processing than prior information.

This information processing bias (i.e., recency) is particularly prevalent when

processing long series of complex, mixed evidence (Hogarth and Einhorn, 1992).

The insolvency case developed for the experiments was based on a going

concern case that proved effective in identifying order/recency bias with experi-

enced accounting professionals (see King, 1989; Arnold and Sutton, 1997). We

selected the going concern case both for its commonalties with the insolvency

decision process and the complexity and thoroughness of the cue information.We modified the case from a going concern decision environment and updated

the case for current economic conditions to provide a better fit with the informa-

tion required for performing an insolvency evaluation.

The revised case was pretested with one insolvency partner, one senior and a

faculty colleague. The case was further revised based on this feedback and pre-

tested again. The second pretest was conducted with a partner and manager

using the intelligent decision aid and with two managers making the decision

without the assistance of the aid. Given supportive feedback from these insolv-

ency practitioners on the realism and appropriateness of the case, the refined

version was used in the experimental sessions.Subjects were provided with three sealed envelopes, clearly labelled for order

of processing. The first envelope (identical for all subjects) contained preliminary

case analysis data and a complete set of financial statements. After reviewing

this information, the instructions asked the subjects to state likelihood estimates

(from 0 to 100) that they would continue to trade-on the client company. These

likelihood estimates formed the initial anchors. The instructions then informed

subjects that one of two colleagues had gathered additional information sup-

porting (or not supporting depending on the case order) a trade-on decision. To

increase the realism of the case, subjects were informed that two different staff

members were given the responsibility to seek the additional evidence. One

https://www.researchgate.net/publication/232326717_Order_Effect_in_Belief_Updating_The_Belief-Adjustment_Model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/232326717_Order_Effect_in_Belief_Updating_The_Belief-Adjustment_Model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

14/26

14 V. Arnold et al.


© AFAANZ, 2004

staff person was asked to accumulate evidence supporting liquidation and the

other to accumulate evidence supporting continued operation (i.e., trade-on).

After reviewing the additional information contained in the second envelope,

the instructions advised subjects to revise their initial likelihood estimates. The

process was then repeated using the other set of additional information gathered

by the other colleague (in the third envelope). The subjects using the intelligent

decision aid completed each segment of the case using the aid to the degree

they wished to do so. Researcher observation and computer log files confirmed

that all aided subjects made extensive use of the aid during completion of the

case study. The three stage sequence of case data and request for likelihood

estimates is typical of the insolvency environment where the decision to trade-

on is made at different stages throughout the engagement; with each new wave

of evidence, the insolvency specialist must make a new assessment. This facil-

itated the use of a realistic end-of-sequence mode (as defined by Hogarth and

Einhorn, 1992) experimental procedure.

The case was designed in a manner that the user must consider conflicting

information provided by the two colleagues and determine how that information

is entered into the intelligent decision aid. Alternatively, raw numbers can be

entered into the aid to perform a what-if analysis. Either way, the user must

decide how to handle the information. The aid also asks for both qualitative and

quantitative estimates at various stages, and the user needs to understand the

problem well enough to provide the estimates. Further, the aid allows the user

to change the weights placed on different information, which is then used by

the intelligent system when formulating its recommendations. All of these situ-ations create the potential for breakdowns in the computer/user interactions,

and hence the possibility for poorer decision-making.

3.2. The intelligent decision aid: INSOLVE

INSOLVE is a computerised intelligent decision aid that has been designed

to replicate the decision-making processes of expert insolvency practitioners

(Leech et al.

, 1998; Collier et al.

, 1999; Leech et al.

, 1999). It is the product of

a four year project funded by the Australian Research Council and the Institute

of Chartered Accountants in Australia (Information Technology Chapter).INSOLVE uses a weighted-additive model where the user can elect to change

the weights assigned to different factors based on subjective judgements of the

specific case circumstances. The model is designed to be a collaborative model

where the user periodically takes control of the process by selecting which

information to enter. Once the user enters the command for the intelligent deci-

sion aid to process, INSOLVE takes control and prompts the user for additional

information that would be useful in making the decision. The user can elect

whether to provide the information – normally based on whether the informa-

tion is available. Once processing has finished, INSOLVE returns a brief report

(generally about two screens long), containing a recommendation together

https://www.researchgate.net/publication/232326717_Order_Effect_in_Belief_Updating_The_Belief-Adjustment_Model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/232326717_Order_Effect_in_Belief_Updating_The_Belief-Adjustment_Model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/227755430_A_generalized_model_of_decision-making_processes_for_companies_in_financial_distress?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/227755430_A_generalized_model_of_decision-making_processes_for_companies_in_financial_distress?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/227755430_A_generalized_model_of_decision-making_processes_for_companies_in_financial_distress?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/232326717_Order_Effect_in_Belief_Updating_The_Belief-Adjustment_Model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/232326717_Order_Effect_in_Belief_Updating_The_Belief-Adjustment_Model?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/227755430_A_generalized_model_of_decision-making_processes_for_companies_in_financial_distress?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

15/26

V. Arnold et al.


© AFAANZ, 2004

with an explanation of its reasoning. At any time, the user can add or change the

information provided and can request a new recommendation from INSOLVE

by selecting the process option.

INSOLVE was developed using 23 insolvency experts (18 partners, 3 man-

agers and 2 bankers) who were willing to participate in the knowledge acquisi-

tion phase (Leech et al.

, 1998). The partners and managers were from the major

professional services firms and accountancy firms specialising in insolvency/

corporate recovery. The resulting cognitive model was validated and refined based

on the feedback of 6 of the experts participating in the knowledge acquisition

phase and subsequently programmed into a working prototype. Formal valida-

tion of INSOLVE was completed with 17 experts – 6 of whom had not been

involved in the knowledge acquisition phase. The 17 experts consisted of 15 firm

partners and directors, one senior manager, and one financial institution senior

manager. The subjects gave the expert system very high scores for validation

(see Leech et al.

, 1998 and Collier et al.

, 1999 for a complete discussion of the

development and validation process).

In summary, the complexity of the model, the collaborative interface, and the

high scores on validation indicate that INSOLVE is a high-level, reliable intel-

ligent decision aid. Hence, INSOLVE is representative of the type of advanced

level intelligent decision aid that would be desired in a practice environment.

3.3. Participants

Several large professional services firms were contacted and asked to pro-vide participants in order to obtain a representative cross-section of high-level

insolvency practitioners (corporate recovery specialists). For subjects using the

intelligent decision aid, a series of training sessions were scheduled in Sydney

and Melbourne. Ten offices of six firms provided a total of 86 experienced

insolvency practitioners. Six of the participants were lost due to computer

memory crashes during the experimental sessions, leaving 80 participants.

These 80 participants each attended a two hour training session conducted at

their firm’s office. The purpose of the session was to introduce the subjects to

INSOLVE. The sessions consisted of a 2:0 minute introduction to INSOLVE,

explanation of the development process, discussion of the system validationresults, and instruction on basic use of the system. After the introductory ses-

sion, the subjects completed a tutorial using the system, which took 35–45 min.

After completing the tutorial and making any inquiries of the roving instructors,

the subjects completed the case.

To obtain additional insolvency practitioners to complete the case without the

aid of INSOLVE, managing partners in offices other than those that participated

in the previously mentioned sessions were contacted. These partners agreed to

distribute the case instruments to their colleagues along with a personal letter

encouraging completion. A total of 170 questionnaires were forwarded to these

partners for distribution. Each mailing contained a number of extra questionnaires

https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/220613984_A_validated_expert_system_for_decision_making_in_corporate_recovery?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

16/26

16 V. Arnold et al.


© AFAANZ, 2004

in hopes that additional subjects might be solicited. Responses were received

from 91 individuals of which 87 were usable. Three were omitted because of

incomplete demographic information prohibiting assessment of experience level,

and one was omitted because the subject failed to provide the initial likelihood

estimate. Using the optimistic assumption that all 170 questionnaires were

distributed, the overall response rate was 54 per cent and the usable response

rate 51 per cent. Responses were returned directly to the researchers in prepaidenvelopes. Demographic information for the subjects is presented in Table 3.

Bédard and Mock (1992) suggest the use of domain specific experience as an

indicator of expertise. Note in Table 3 that the promotion from senior to man-

ager places the insolvency practitioner in a position of primary responsibility

for making the insolvency decision most of the time. A distinct gap in know-

ledge and expertise between staff/seniors and managers/partners was also

evident during the knowledge acquisition phase in the development of INSOLVE.

Thus, there is a major difference in task domain experience in the present

study’s participants. As long as the company in financial distress continues to

operate, this decision must be reassessed on a continuous basis throughout theengagement. Additionally, under Australian law, in many engagements the CA

firm absorbs the risk of any future additional losses incurred by the insolvent

company if the CA chooses to trade-on. On the other hand, the CA continues

to collect hourly fees if the company continues to trade-on. If the partner or

manager were not expert, they would risk incurring huge losses rather quickly

and/or lose the confidence of existing and potential clients. As a result, the part-

ners and managers were deemed to be experts at the given task.

The experienced staff and seniors are considered novices despite their aver-

age 3.67 years of insolvency experience. At this level, the focus tends to be on

financial statement analysis and evidence collection, and not so much on actually

Table 3

Demographic information

Variable Unaided decision-makers Aided decision-makers Combined

Position:

No. staff /seniors 39 43 82

No. managers/partners/directors 48 37 85

Total 87 80 167

Age in Years

Staff/seniors 25.26 25.67 25.48

managers/partners/directors 35.81 35.58 35.29

Insolvency experience in Years

Staff/seniors 3.33 3.67 3.51

Managers/partners/directors 12.10 10.26 11.30

Primary decision-making responsibility

Staff/seniors 4.36 2.16 3.06Managers/partners/directors 63.56 68.19 65.58

https://www.researchgate.net/publication/27828256_Expert_and_novice_problem_solving_behaviour_in_audit_planning_An_experimental_study?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=https://www.researchgate.net/publication/27828256_Expert_and_novice_problem_solving_behaviour_in_audit_planning_An_experimental_study?el=1_x_8&enrichId=rgreq-1f23b7d8-f937-4340-bed8-be0522bfea20&enrichSource=Y292ZXJQYWdlOzQ3Mzc3MzU7QVM6MTU2OTA4NjE5MDUxMDEzQDE0MTQ0MjEzMzg2NzA=

8/18/2019 SSRN-id513260

17/26

V. Arnold et al.


© AFAANZ, 2004

making the decision to liquidate or trade-on. Hence, despite their significant

experience in years, this experience is differentiated from the task specific

experience that would develop high levels of expertise for this particular decision.

4. Experimental results

The data collected from the different experimental sessions provide the basis

for analysing the two alternative propositions from the theory of technology

dominance. The data for the first experiment and the related hypotheses (H

1

–H

2

)

testing are based on cases completed by novices, while the second experiment

and the related hypotheses (H

3

–H

4

) testing are based on experts.

4.1. Technology dominance of novices (H

1

–H

2

)

The first experiment focuses on the potential business decision risk thatnovice decision-makers will make significantly worse decisions when using an

intelligent decision aid, resulting in the presence of information processing

biases

.

H

1 and H2 isolate the two underlying phenomena necessary for such a

significant business decision risk effect to exist.

The purpose of H1 is to test whether the order of the information induced dif-

ferences in the decision of novice decision-makers using the intelligent decision

aid. An ANOVA (analysis of variance) is used to test for differences in decisions

as a result of the order of information presentation (H1). These effects are

examined using the change in likelihood estimate that the company would con-

tinue to trade-on from the initial decision to the final decision as the dependentvariable, and the dichotomous variable, order, as the independent variable. An

examination of the data shows that the average change in likelihood estimate

for the group receiving information supporting a trade-on first is a decrease of

43.3 while the average change for the second group which received the evidence

supporting liquidation first is a decrease of only 6.8. The results in Table 4

Table 4

Results from novices with decision aid

Panel A: Test for order effects

Degrees of freedom F-statistic Significance

Between groups 1 26.287

8/18/2019 SSRN-id513260

18/26

18 V. Arnold et al. / Accounting and Finance 44 (2004) 1–26

© AFAANZ, 2004

Panel A demonstrate a highly significant order effect for the users of the

intelligent decision aid ( p < 0.001).

In order to test whether the order effects are indicative of a recency bias, the

absolute value of the change in likelihood estimate from the information pre-

sented first is compared to the information presented last. If a recency effectoccurs, the average change resulting from the second revision should be signific-

antly greater than the average change resulting from the first revision. A t -test

is conducted to test for differences and the results, presented in Table 4 Panel

B, indicate an overall recency effect ( p = 0.001). The average change as a result

of the first revision is about 23 points while the change as a result of the last

revision is about 40 points. This recency effect is also reflected in Figure 1 by

the classic fishtail pattern associated with a recency induced information order

bias.

The purpose of H2 is to test whether use of the intelligent decision aid

exaggerates the impact of order on the decision-making processes of noviceinsolvency practitioners. Further inspection of Figure 1 shows that the width of

the fishtail increased for the users of the intelligent decision aid indicating an

increase in order effects for the novices using INSOLVE. ANOVA is used to

test for the impact of intelligent decision aid an

Documents

SSRN-id513260