AN AUDIT PARTNER-LED FIELD INTERVENTION IN FRAUD BRAINSTORMING

Sean Dennis

PhD Candidate

University of Wisconsin – Madison School of Business

sadennis@bus.wisc.edu

Karla M. Johnstone

EY Professor

University of Wisconsin – Madison School of Business

975 University Avenue

Madison, WI 53706

kjohnstone@bus.wisc.edu

June 1, 2014

We thank participants at research workshops at the University of Wisconsin-Madison, University

of Notre Dame, University of Missouri – Columbia, University of Connecticut, and NEBARS.

We express particular appreciation to comments from Jean Bedard, Jere Francis, Jeremy Griffin,

Brian Mayhew, Nate Newton, Dave Ricchiute, Terry Warfield, and Arnie Wright. We thank the

sponsoring audit firm for monetary support of this project, and we thank leadership and

participants at two other audit firms. Johnstone acknowledges support through her professorship

with EY, as well as the Andersen Center for Financial Reporting at the University of Wisconsin

School of Business. Dennis acknowledges financial support from the Accounting Doctoral

Scholars Program. Johnstone also acknowledges helpful comments of Noel Harding and Ken

Trotman during her Visiting Professorial Fellow sabbatical at the University of New South

Wales. Finally, we express our appreciation to Eric Condie and Amy Tegeler, who served as

diligent coders of qualitative data.

AN AUDIT PARTNER-LED FIELD INTERVENTION IN FRAUD BRAINSTORMING

SUMMARY: In a field experiment, we manipulate guidance to audit partners through an

intervention intended to affect the approach they take in leading fraud brainstorming sessions for

actual audit engagements. We examine how this audit partner-led field intervention is associated

with processes and outcomes of these sessions. We predict and find associations between the

intervention and some, but not all, of our process and outcome measures. Analyses of

quantitative data suggest the intervention improves processes and outcomes related to fraud risk

factor identification, but not those related to planned fraud risk responses. Analyses of qualitative

data reveal associations between the intervention and attributes of planned fraud risk responses,

but not the types of fraud risk factors that were identified. This suggests that while the fraud risk

profile of clients in the field does not differ by experimental condition, the audit responses to

these circumstances do differ by experimental condition.

Keywords: Audit planning, field experiment, fraud brainstorming, professional skepticism.

AN AUDIT PARTNER-LED FIELD INTERVENTION IN FRAUD BRAINSTORMING

I. INTRODUCTION

Although audits are not necessarily designed to detect fraud, regulators and users continue to

demand quality in auditors’ fraud detection capabilities (see e.g., Carmichael 2004; Hammersley

2011). Professional standards require auditors to conduct a fraud brainstorming session as part of

each audit (AICPA 2002b), illustrating the importance of this task. The PCAOB has criticized

auditors with regard to the quality of brainstorming and has expressed concern that auditors lack

appropriate professional skepticism in this task (PCAOB 2007, 4). Prior research reports

considerable variation in the extent to which auditors emphasize skepticism and deploy resources

in brainstorming (Bellovary and Johnstone 2007; Brazel et al. 2010). Recent research illustrates

the important role that the audit partner plays in brainstorming (e.g., Carpenter and Reimers

2013; Gissel 2013). The purpose of this study is to examine how an audit partner-led field

intervention affects the processes and outcomes of fraud brainstorming sessions.

The term “tone at the top” emerged in the field of auditing to describe organizational

leadership with respect to internal control over financial reporting. COSO (2011, p. 27) notes

that leadership affects the control environment in that “management and the board of directors or

equivalent oversight body are expected to lead by example”. COSO (2011, p. 28) goes on to state

that “tone is impacted by the personal conduct of management…”. We refine this focus to

investigate the role of audit partner engagement-level leadership. We develop an intervention

intended to improve partners’ leadership by facilitating a fraud brainstorming session that

emphasizes productive interpersonal interaction, motivates effectiveness and efficiency, and

promotes professional skepticism. The audit partner-led field intervention (hereafter, “the

2

intervention”) is a memo to the audit partner with specific, actionable suggestions that include

the following instructions:1

Emphasize the session as a training

opportunity.

Discuss the importance of effective

and efficient fraud brainstorming.

Discuss the importance of professional

skepticism targeted at specific accounts

with a potentially higher level of fraud

risk.

Emphasize both effectiveness and

efficiency to promote an

appropriately calibrated response to

fraud risk.

Discuss the importance of professional

skepticism in general throughout the audit.

Prior research provides evidence relevant to the role of the audit partner in fraud

brainstorming sessions using hypothetical cases in laboratory settings. In an early study,

Carpenter (2004) finds that teams spend more time brainstorming when the audit partner

emphasizes effectiveness as opposed to efficiency. Carpenter and Reimers (2013) find that fraud

risk assessments are higher when the partner emphasizes professional skepticism and that fraud-

related audit procedures are only responsive to fraud risk when the partner emphasizes

professional skepticism. Additionally, Gissel (2013) finds that audit staff are more willing to

share private information related to fraud risks during brainstorming when the audit partner

creates an atmosphere that is viewed as “psychologically safe,” which leads to more accurate

fraud risk assessments. We extend this research by investigating whether an intervention can

affect the approach partners take in leading an actual brainstorming session for an audit

engagement in the field and how that intervention affects brainstorming processes and outcomes.

We conducted the field experiment in conjunction with, and immediately following, the

fraud brainstorming sessions of a sample of 77 audit engagements conducted from July 2013

1 AU 316, Consideration of Fraud in a Financial Statement Audit (formerly SAS No. 99) does not explicitly require

audit partners to perform all of these actions (AICPA 2002b). We developed these instruction in collaboration with

senior leadership at the sponsoring audit firm. These individuals have significant experience with audit firm

methodology and training. The specific content of the intervention was designed to emphasize best practices in a

concise manner that facilitates effective implementation of professional standards and the firm’s methodology.

3

through January 2014 at three audit firms (two Big 4 firms and one international firm). Each

engagement team in the sample was randomly assigned to either the treatment condition (N = 37

partners) or the control condition (N = 40 partners). Each partner in the study received a memo

informing him/her that his/her audit engagement will be involved in a brainstorming research

study; the memo in the treatment condition also included the intervention. After the

brainstorming sessions occurred, a designated audit firm contact person notified the audit

manager and senior on each engagement that their engagement was involved in the study. The

manager and the senior each received a survey to complete individually. The survey measures

audit managers’ and seniors’ perceptions of the topics partners discussed and the issues partners

emphasized during brainstorming, the processes of the session, and the outcomes of the session,

along with descriptive details about the sessions. We received 75 and 73 completed surveys from

managers and seniors, respectively. We expect the intervention to be positively associated with

the quality of brainstorming processes and outcomes.2

The results show that audit managers’ and seniors’ perceptions of the topics partners

discussed and the issues they emphasized during fraud brainstorming differ in certain respects by

experimental condition. In the treatment condition, there is a greater likelihood that the audit

partner discussed his/her prior experiences with fraud during brainstorming, a greater likelihood

that the partner addressed the issues of effectiveness and efficiency to promote an appropriately

calibrated response to fraud risk, and a greater likelihood that the partner discussed the

importance of professional skepticism with respect to specific accounts on the engagement that

have a higher level of fraud risk, compared to in the control condition. However, in both the

2 Note that some partners may exhibit the behaviors described in the intervention even if they are in the control

condition. Therefore, it is an empirical question as to whether there will be significant differences across

experimental conditions in manager/senior perceptions of the approach audit partners take during brainstorming with

respect to topics they discuss and the issues they emphasize.

4

treatment and control conditions, there were about equal levels of the extent to which the partner

emphasized brainstorming as a training/professional development opportunity, about equal levels

of the extent to which the partner discussed the importance of effective and efficient

brainstorming in general, and about an equal likelihood that the partner discussed the importance

of professional skepticism in general throughout the audit. We interpret this as evidence that

partners incorporate these latter three discussion topics and issues of emphasis in brainstorming

sessions in practice as a matter of professional routine.

The multivariate hypothesis-testing results reveal the intervention is associated with

several fraud brainstorming processes. Consistent with expectations, the intervention is

associated with greater increases in self-assessed manager and senior professional skepticism

both in general and with respect to specific accounts with a higher level of fraud risk. Also

consistent with expectations, the intervention is associated with a greater extent of discussion

about how management might perpetrate fraud, as well as longer brainstorming sessions (by

about ten minutes on average); however, we find no association between the intervention and the

extent of discussion about audit responses to fraud risk.

We also analyze quantitative and qualitative measures of fraud risk identification

outcomes and fraud risk response outcomes. Overall, inferences related to fraud brainstorming

outcomes complement the inferences related to brainstorming processes. The intervention is

associated with quantitative measures of fraud risk factor identification outcomes, but not

qualitative measures of fraud risk factor identification outcomes. Specifically, we predict and

find that the intervention is associated with the identification of more fraud risk factors overall

and more new fraud risk factors; this complements the positive association between the

intervention and the extent of discussion about how management might perpetrate fraud.

5

However, we find no association between the intervention and the percentages of fraud risk

factors that relate to revenue recognition or management override of controls, respectively.

Therefore, the fraud risk profile of clients in the field does not differ by experimental condition.

Inferences concerning fraud risk response outcome measures similarly complement the

inferences from the analyses of process measures. Specifically, we do not find any of the

predicted associations between the intervention and quantitative measures related to planned

audit procedures, including no association between the number of planned procedures, the

number of new planned procedures, or the number of planned procedures intended to incorporate

an element of unpredictability to respond to fraud risk. Also in contrast with expectations, the

intervention is associated with a lower likelihood of tailoring the audit plan to the current year

audit by eliminating fraud risk responses used in prior years. The lack of predicted associations

between the intervention and quantitative measures of planned audit procedures result

complements the lack of an association between the intervention and the extent of discussion

during brainstorming about audit responses to fraud risk. These results are unexpected, but

consistent with prior literature that shows auditors often have difficulty linking evaluations of

fraud risk with appropriate fraud risk responses (see, e.g., Hammersley 2011), and with

substantive test modifications in other audit tasks (Mauldin and Wolfe 2014).

Interestingly, we do find associations between the intervention and qualitative

characteristics of fraud risk response outcomes. Specifically, the intervention is positively and

marginally significantly associated with percentage of fraud risk responses that relate to the

nature of planned audit procedures. Additionally, the intervention is negatively associated with

the percentages of fraud risk responses that relate to the extent and timing, respectively, of

planned audit procedures. These results suggest that, compared to engagement teams in the

6

control condition, engagement teams in the treatment condition focused more on what was being

done to respond to fraud risk and less on how much was being done and when it was being done.

Results for control variables reveal several notable insights. Higher levels of audit partner

experience on the client are associated with decreases in self-assessed manager and senior

professional skepticism during the brainstorming session, less discussion about how management

may perpetrate fraud, the identification of fewer new fraud risk factors during the session, more

discussion about potential management override of controls, and a reduced willingness to tailor

the audit plan by eliminating fraud risk responses used in the prior year audit. Thus, there

appears to be some inertia with respect to longer audit partner tenure on an engagement that

yields “stickiness” in some processes and outcomes of brainstorming sessions, which has

interesting implications for research on partner tenure (e.g., Bedard and Johnstone 2010). Results

also reveal greater manager experience on the client is associated with more discussion about

how management might perpetrate fraud and more discussion about audit responses to fraud

risks, suggesting possible benefits to continuity at the manager level. Similarly, the engagement

team’s level of expertise on the client is associated with more discussion about how management

might perpetrate fraud and more discussion about audit responses to fraud risks.

Our results extend prior research that shows the benefits of interactive brainstorming

(e.g., Carpenter 2004; Carpenter 2007; Hoffman and Zimbelman 2009; Lynch et al. 2009; and

Carpenter et al. 2011), auditing-specific interventions intended to facilitate brainstorming (e.g.,

Carpenter 2004; Hoffman and Zimbelman 2009; Lynch et al. 2009; Trotman et al. 2009;

Carpenter and Reimers 2013; and Gissel 2013), and brainstorming session quality (Brazel et al.

2010). This study also extends prior research that seeks to understand, and ultimately mitigate,

the potential drawbacks of brainstorming in interactive groups (e.g., Chen et al. 2013).

7

A particularly important contribution of this study is that no prior research in this area

uses a field experiment design in which an intervention manipulates the behavior of audit

partners while performing an actual audit engagement in the field. The managers and seniors

participating in our study report a mean of 30 months of experience on the clients involved in

this field experiment. These individuals likely develop rich bodies of client-related knowledge

through this experience that they can attend to throughout fraud brainstorming sessions.

Moreover, institutions in practice create powerful incentives for auditors to take brainstorming

seriously. These factors enable enhanced experimental realism in the current study, relative to

prior fraud brainstorming studies (Swieringa and Weick 1982).

Taken together, our results extend prior research and provide important insights to

practice by showing that a field intervention can improve the approach audit partners take in

leading fraud brainstorming sessions in the field and that the use of such an intervention is

associated with improvements in some important brainstorming processes and outcomes. We

therefore recommend that regulators and practitioners offer explicit guidance on connecting

brainstorming sessions to audit program design, such as through simple, actionable interventions.

This paper proceeds as follows. Section 2 provides the literature review and hypotheses. Section

3 describes the method. Section 4 reports results and Section 5 concludes.

II. LITERATURE REVIEW AND HYPOTHESES

Non-experimental Field Research in Fraud Brainstorming

Two prior studies using field data report evidence of considerable variation in fraud

brainstorming processes and outcomes. Bellovary and Johnstone (2007) interviewed 22 auditors

at all personnel levels at seven audit firms (including all Big 4 firms) to provide descriptive

evidence about how auditors conduct brainstorming sessions. They note variability in the level of

8

contribution to the session among participants, formatting of the session, use of audit firm

guidance, and leadership. Additionally, they note that auditors sometimes view the sessions as

training opportunities for inexperienced team members. Brazel et al. (2010) conduct a field

survey using 179 audit engagements at all four Big 4 firms and one international firm to develop

a measure of brainstorming session quality. They find considerable quality variation in practice.

In addition, their results show that quality brainstorming improves the relationship between fraud

risk factors and fraud risk assessments.

Experimental Research in Fraud Brainstorming

Table 1 provides a summary of experimental research on fraud brainstorming. Each of

these experiments (except our own) uses a hypothetical case, often based on a fraud that occurred

in practice. Panel A categorizes this research based upon differences in experimental designs,

while Panel B summarizes dependent measures. Panel A illustrates the variation in brainstorming

research examining the guidance that participants receive, from no specific mention of

“brainstorming” per se, to brainstorming without explicit guidance, to brainstorming with

auditing-specific interventions intended to facilitate the process.3

INSERT TABLE 1 HERE

Two studies examine associations between general fraud brainstorming guidance and

brainstorming outcomes. Trotman et al. (2009) compare three different forms of group

discussion using 111 experienced auditors: (1) interacting in a team that receives no mention of

the term “brainstorming” and no explicit guidelines, (2) interacting in a team that receives

3 Several prior experimental studies compare fraud brainstorming processes and outcomes using different

communication formats, such as individuals working independently, individuals working in nominal groups, and

individuals working in interacting groups (e.g., Carpenter 2004; Carptenter 2007; Hoffman and Zimbleman 2009;

Lynch et al. 2009; Carpenter et al. 2011; Chen et al. 2013). In the current discussion, we focus on findings related to

guidance provided in fraud brainstorming situations, including auditing-specific interventions intended to facilitate

brainstorming. We do not manipulate interaction format; as discussed subsequently, approximately 22 percent of the

participants in our study reported using a formal interaction format and we control for the use of a formal format in

our hypothesis-testing models.

9

explicit brainstorming guidelines based on Osborn (1957), and (3) interacting in a team that

receives “pre-mortem” instructions modeled after those in Klein (1999).4 The results show that

teams brainstorming with explicit guidelines and pre-mortem teams listed more potential frauds

(quantity) and more “expert-identified frauds” (quality), compared to teams that receive no

explicit brainstorming guidelines.

Additionally, Hoffman and Zimbelman (2009) provide evidence related to whether

brainstorming helps auditors change both the nature and extent of their audit work. In an

experiment with 91 audit managers, the authors manipulate whether or not auditors work alone

with no mention of the word “ brainstorm” or in a three-person team with specific instructions

related to brainstorming. The results show that team brainstorming is associated with effective

modifications to a list of standard audit procedures for accounts receivable. Collectively, these

two studies illustrate that brainstorming guidelines can improve fraud brainstorming outcomes.

In an early paper examining fraud brainstorming, Carpenter (2004) tests interventions

related to audit partner emphasis on efficiency versus effectiveness. In an experiment with 240

auditors (80 managers, 80 seniors, and 80 staff), she finds that when the audit partner emphasizes

efficiency (as opposed to effectiveness) audit teams spend less time in fraud brainstorming (a

process measure). Carpenter and Reimers (2013) extend Carpenter (2004) by examining the

association between audit partner emphasis on professional skepticism and brainstorming

outcomes. Using 80 audit managers, they manipulate partner emphasis on professional

skepticism in general during brainstorming (high or low). The results show fraud risk

assessments are higher when the partner emphasizes professional skepticism. In addition, the

4 Pre-mortem instructions direct the auditors to assume a backward-thinking perspective. Participants receiving the

pre-mortem manipulation in Trotman et al. (2009) are told to imagine the following scenario: “It is months into the

future, the audit has already been completed, and no material fraud was uncovered. However, it has just been

announced in the press that there has been a material financial reporting fraud at the company. The manager and

partner have given you no details of the nature of the fraud” (p. 1121).

10

results show auditor choice of fraud-related audit procedures is only responsive to fraud risk

assessments when fraud is present and the partner emphasizes professional skepticism.

Gissel (2013) examines the effect of psychological safety, as created by the audit partner,

in fraud brainstorming.5 In an experiment with 67 staff and senior auditors, Gissel (2013)

manipulates psychological safety (safe or unsafe) and the presence of a general professional

skepticism intervention using videos of a partner and a manager in a simulated brainstorming

session. The results show auditors are more willing to share private information related to fraud

risks during a brainstorming session when the audit partner’s behavior creates an atmosphere that

is viewed by staff and seniors as psychologically safe. Greater willingness to share private fraud

information then is associated with more accurate fraud risk assessments. In contrast to

expectations, the general professional skepticism intervention does not affect willingness to share

information or fraud risk assessments.

Two other studies use experimental settings to examine fraud brainstorming interventions

that are not specific to audit partner behaviors. Hoffman and Zimbleman (2009) find that an

intervention intended to prompt strategic reasoning is associated with effective modifications to

the standard audit procedures.6 Additionally, Lynch et al. (2009) examine the effect of a content

facilitation intervention, which consists of prompts that address issues emphasized in AU 316. In

an experiment with 188 auditing students, they find a content facilitation intervention is

associated with the identification of a larger number of relevant fraud risk factors.

The Intervention

We extend prior studies that employ interventions intended to aid the efficacy of fraud

brainstorming by employing an intervention that attempts to influence the approach audit

5 Gissel defines psychological safety as “a sense of being able to show and employ self without fear of negative

consequences to self-image, status, or career.” 6 Hoffman and Zimbleman (2009) do not find a significant interaction between their two manipulations.

11

partners take in leading an actual fraud brainstorming session for an audit engagement in the

field. In order for the intervention to be effective, the following conditions must be satisfied: (1)

the instructions in the intervention must relate to audit partner behaviors where there is room for

improvement in practice, and (2) members of the audit engagement team must respond to the

behaviors of the audit partner in the experimental condition.

Practitioner-oriented articles suggest the auditing profession has a collective interest in

improving fraud detection (e.g., Landis et al. 2008; Wood and Pickerd 2011). The PCAOB has

criticized audit teams with respect to how they conduct brainstorming sessions, particularly with

respect to a perceived lack of professional skepticism (PCAOB 2007), and prior research

provides some descriptive evidence consistent with the PCAOB’s views (e.g., Bellovary and

Johnstone 2007; Brazel et al. 2010). Collectively, these factors suggest room for improvement in

the way auditors conduct fraud brainstorming; if the elements in the intervention relate to such

improvement opportunities, then the first condition will be satisfied.

Regarding the second condition, prior research suggests managers and seniors attend and

respond to differences in partner behaviors during fraud brainstorming (Carpenter 2004;

Carpenter and Reimers 2013; Gissel 2013). In prior studies, manipulations related to audit

partner behaviors during brainstorming are delivered directly to subordinates via experimental

instruments. In contrast, we manipulate audit partner behavior in the field during brainstorming

sessions on real engagements and measure subordinates’ perceptions of audit partner behavior

subsequent to the sessions. It seems reasonable to expect that these subordinates will respond to

partner behavior. Still, it is an empirical question as to whether the intervention will successfully

manipulate the behaviors that audit partners exhibit in brainstorming sessions, as proxied by

engagement team members perceptions of the topics partners discuss and issues partners

12

emphasize. We therefore include a series of questions in the survey to determine whether

managers and seniors perceive the audit partner behaviors referenced in the intervention

differently, depending on experimental condition. We use the managers’ and seniors’ responses

to these questions to perform a manipulation check and to provide evidence on the behaviors that

audit partners exhibit in fraud brainstorming sessions as a matter of professional practice.

Fraud Brainstorming Processes

AU 316, Consideration of Fraud in a Financial Statement Audit (formerly SAS No. 99),

discusses the importance of maintaining professional skepticism throughout the audit (AICPA

2002b). Notably, this standard discusses professional skepticism in general throughout the audit

and professional skepticism with respect to accounts with a higher level of fraud risk. Given the

importance of professional skepticism in fraud detection and following Carpenter (2004), we

expect that increases in professional skepticism, both in general and with respect to accounts

with a higher level of fraud risk, represent high quality processes in fraud brainstorming sessions.

Specifically, we argue that increases in professional skepticism during brainstorming sessions

facilitate improved discussion as sessions progress.

We also investigate process-oriented dependent measures related to the extent of certain

discussions during the fraud brainstorming session. In order for auditors to respond to fraud risk

effectively and in a way that complies with the applicable audit standards, they must both

identify relevant fraud risks and develop effective audit responses to those risks. Prior research

on brainstorming suggests that quantity drives quality in terms of idea generation (e.g., Trotman

et al. 2009; Osborn 1957). Therefore, we argue that more extensive discussion about how

management might perpetrate fraud and more extensive discussion about potential audit

responses to fraud represent high quality brainstorming processes.

13

A recent working paper provides a rich contextual understanding for how interventions

can facilitate improvements in fraud brainstorming processes and outcomes. Chen et al. (2013)

focus on three potential drawbacks of interactions in brainstorming: production blocking,

evaluation apprehension, and social loafing. They argue that the electronic brainstorming setting

minimizes production blocking (due to a lack of interruptions) and evaluation apprehension (due

to anonymity), thereby enabling a more precise examination of the effect of social loafing on

brainstorming outcomes. In an experiment with 111 audit seniors and managers, they find

evidence consistent with social loafing of seniors driving the differences between nominal

groups and interacting teams in fraud hypothesis generation. Notably for the current study, these

findings suggest audit partners can improve brainstorming outcomes by mitigating potential

drawbacks in brainstorming, such as social loafing, through their leadership during the session.

The limited prior research on associations between fraud brainstorming processes and

audit-specific brainstorming interventions provides mixed evidence regarding the effectiveness

of these interventions. On the one hand, Carpenter (2004) finds the predicted association

between the time spent brainstorming and an intervention emphasizing either effectiveness or

efficiency. Additionally, Gissel (2013) finds the predicted association between participants’

willingness to share private information and a psychological safety manipulation, but not a

professional skepticism-related intervention.

In field settings, partners can exert significant influence over audit activities. Notably for

the current study, partners can focus and re-direct discussions during fraud brainstorming

sessions. Partners can also take measures to address issues of evaluation apprehension and social

loafing by seniors (e.g., Kerr and Tindale 2004; Chen et al. 2013). Live interactions in the field

enable a dynamic implementation of the intervention; such an experimental manipulation would

14

be prohibitively difficult or costly in a laboratory setting. These factors contribute to the

enhanced experimental realism in the current study (e.g., Swierenga and Weick 1982) and

increase the likelihood of associations between the intervention and fraud brainstorming

processes. Following this discussion, we posit the following hypothesis:

Hypothesis 1. The intervention is positively associated with indicators of the quality of

fraud brainstorming session processes.

Fraud Brainstorming Outcomes

We consider brainstorming outcomes in terms of both fraud risk factors and fraud risk

responses, consistent with prior research (e.g., Carpenter 2004; Carpenter 2007; Lynch et al.

2009; Trotman et al. 2009; Carpenter et al. 2011; Carpenter and Reimers 2013; and Chen et al.

2013). We also separately identify the number of fraud risk responses intended to incorporate an

element of unpredictability in the audit as a fraud brainstorming outcome. Further, since all

engagements in our sample are continuing clients, the quantity of fraud risk factors and fraud risk

responses may be similar to that from the prior year. We therefore expect that the number of new

fraud risk factors and new fraud risk responses represent brainstorming outcomes that will be

positively associated with the intervention. This approach is consistent with Hoffman and

Zimbleman (2009), who analyze audit program modifications as a brainstorming outcome.

The PCAOB has expressed concerns about “mechanical” implementation of AU 316

(PCAOB 2007). If audit teams implement this guidance mechanically, then the audit plan will

lack appropriate tailoring. Moreover, if audit teams do not critically evaluate fraud audit

programs each year, then fraud audit programs will become inappropriate as conditions change.

We therefore expect that the elimination of fraud risk responses that had been used in prior year

15

audits (i.e., tailoring the fraud audit program to the current year audit) represents a brainstorming

outcome that will be positively associated with the intervention.

Prior research that investigates associations between interventions and fraud

brainstorming outcomes generally finds different inferences related to fraud risk factors, as

compared to fraud risk responses (see, e.g., Hammersley 2011). Several prior studies find

associations between interventions and either improved fraud risk factor identification or

improved fraud risk assessments, which in turn reflect improved risk factor identification (e.g.,

Lynch et al. 2009; Carpenter and Reimers 2013; Gissel 2013). However, a growing number of

studies suggest auditors have difficulty linking evaluations of fraud risk with appropriate fraud

risk responses (e.g., Asare and Wright 2004; Mock and Turner 2005; Hammersley et al. 2011).

Hammersley (2011, 118), in particular, notes “effective changes to planned procedures in

response to perceptions of increased fraud risk is a joint test of whether [1] the fraud risk factor

identified is useful for this purpose, [2] auditors recognize that the procedures should be

modified, [3] auditors know which procedures should be modified, and [4] auditors know how to

modify the procedure appropriately.” If the intervention does not promote all four conditions

necessary for audit teams to make effective modifications to planned procedures, then the

intervention will not be associated with the brainstorming outcome variables. Moreover, one

dimension of the Brazel et al. (2010) measure of brainstorming quality that is particularly

relevant to the relationship between fraud risk assessments and fraud risk responses is the extent

of discussion about audit responses to fraud risk. If we find no association between the

intervention and this process variable, then it is possible that we will correspondingly find no

association between the intervention and the fraud risk response-related outcome variables.

16

On the other hand, there are two important reasons to expect associations between the

intervention and fraud risk response-related outcome variables in the current field experiment.

First, we include instruction points and illustrative examples in the intervention that are relevant

to each of the elements identified by Hammersley (2011). If the intervention helps engagement

teams satisfy these necessary conditions, then the likelihood of associations between the

intervention and fraud risk response-related outcomes will increase. Additionally, Brazel et al.

(2010) find that high quality brainstorming, measured using process variables, improves the

relationships between fraud risk factors and fraud risk assessments and between fraud risk

assessments and fraud risk responses. Therefore, if the intervention improves fraud

brainstorming processes, as predicted in H1, then the intervention will also improve fraud

brainstorming outcomes (e.g., Gissel 2013). Following this, we posit the following hypothesis:

Hypothesis 2. The intervention is positively associated with indicators of the quality of

fraud brainstorming session outcomes.

In addition to the preceding quantitative measures of fraud brainstorming outcomes, we

analyze qualitative characteristics of fraud risk factors and fraud risk responses. Specifically, we

examine whether the intervention is associated with the identification of higher percentages of

risk factors related to revenue recognition and management override of controls, respectively (as

these two areas receive particularly direct emphasis in AU 316). We perform these analyses to

provide insight on whether the intervention perhaps promotes incremental focus on client-

specific fraud-related risks. We further examine whether the intervention is associated with the

percentages of fraud risk responses for a given risk factor that relate to the nature, timing, or

extent, respectively, of planned procedures. We perform these analyses to provide insight into

whether the intervention is associated with what is being done to respond to fraud risk, how

17

much is being done, and/or when these responses occur. We make no predictions related to the

association between these qualitative variables and treatment effects.

III. METHOD

Participants and Research Design

We conducted the study in conjunction with, and immediately following, the fraud brainstorming

sessions of a sample of 77 audit engagements from July 2013 through January 2014.7 Three audit

firms (two Big 4 firms and one international firm) participated in the study, with relative

contribution levels of 77 percent, 16 percent, and seven percent, respectively.8 Each engagement

team in the sample was randomly assigned to either the treatment condition (N = 37 audit

partners) or the control condition (N = 40 audit partners).9 Each partner in the treatment

condition, TREATMENT = 1, received a memo informing him/her that his/her audit engagement

will be involved in a fraud brainstorming research study; this memo also included the

experimental intervention. Each partner in the control condition, TREATMENT = 0, received a

memo informing him/her that his/her engagement would be involved in a fraud brainstorming

research study; necessarily, this memo did not contain the experimental intervention.10

Figure 1

7 As previously noted, all engagements in the sample relate to continuing clients; none of the engagements in the

sample are new to the respective audit firm. 8 The distribution of Treatment (Control) condition observations across audit firms is as follows: 55 (59), 12 (12),

and 4 (6) observations relate to Firm A, Firm B, and Firm C, respectively. The authors obtained institutional review

(i.e., human-subjects) approval for the study, and all participants were given the option to not complete the study. 9 We visited each participating audit firm office to meet with the senior audit partner that served as the “local office

champion” for this research study (e.g., business unit managing partner or lead area technical partner) and the

individual that would serve as the local office contact person. During the meeting, we delivered the experimental

materials and trained both individuals on the process of administering the experiment. We emphasized the

importance that the respective “champion” randomly select and randomly assign the engagements to the

experimental conditions. The individuals assured us that they would adhere to the administration, random selection,

and random assignment processes diligently. We have no reason to believe that there were any systematic biases in

the selection or assignment of engagements. However, due to client confidentiality constraints, we were unable to

oversee the actual selection or assignment processes. We articulate this issue further in the conclusion. 10

We printed all memos on audit firm letterhead. Memos were addressed from a prominent senior partner in the

participant’s specific business unit or region (e.g., business unit managing partner or lead area technical partner).

18

Panel A displays the wording of the memo provided to partners in both the treatment and control

conditions, and Figure 1 Panel B depicts the research design and project logistics.

INSERT FIGURE 1 HERE

After the fraud brainstorming session, one audit manager and one audit senior on each

engagement were notified about participating in the study. Therefore, the managers and seniors

were not aware that partners received experimental instructions prior to the brainstorming

session, nor were they aware during brainstorming that the session would be a part of a research

study.11

These individuals then completed a survey requesting information about the client, the

brainstorming process, and various associated outcomes. A total of 75 managers and 73 seniors

completed the surveys.12

The authors coded the quantitative data from the surveys. Two second-

year PhD students performed the coding of the qualitative data from the surveys.13

Manipulation Check

Table 2 contains comparisons, between experimental conditions, of managers’ and

seniors’ perceptions of indicators of the audit partner fraud brainstorming behaviors. We use

these comparisons as a manipulation check. TRAINING_OPP measures the extent to which the

audit partner emphasized fraud brainstorming as a training/professional development opportunity

on a scale from 1 (low emphasis) to 10 (high emphasis). PTR_EXPERIENCES is a dichotomous

variable equal to one if the audit partner discussed his/her prior experiences with fraud during

brainstorming. EFFECTIVE_EFFICIENT measures the extent to which the audit partner

discussed the importance of effective and efficient brainstorming on a scale from 1 (no

11

From an institutional review perspective, the experimental instructions provided to the partners are conceptually

identical to other firm-sponsored partner-only training programs of which managers and seniors are also unaware. 12

The cover letter on the survey instruments states “You may refer to workpapers and your notes when completing

the survey.” We did not measure the extent to which participants consulted these materials; however, several

respondents included notes from the session and/or sections of audit programs with their survey responses. 13

Each of these individuals previously held a Senior Manager position at a Big 4 Firm, with eight years and 12

years, respectively, of public accounting experience. Inter-rater reliability among the coders was 89%.

19

discussion) to 10 (significant discussion). CALIBRATED_RESPONSE is a dichotomous variable

equal to one if the audit partner addressed the issues of effectiveness and efficiency to promote

an appropriately calibrated response to fraud risk; equals zero otherwise. PS_SPECIFIC is a

dichotomous variable equal to one if the audit partner discussed the importance of professional

skepticism with respect to specific accounts on the engagement with a higher level of fraud risk;

equals zero otherwise. PS_GENERAL is a dichotomous variable equal to one if the audit partner

discussed the importance of professional skepticism in general throughout the audit; equals zero

otherwise. See Appendix A for variable definitions.

INSERT TABLE 2 HERE

The results show that managers’ and seniors’ perceptions do, indeed, differ by

experimental condition. Partners in the treatment condition (as compared to the control

condition) more often discussed prior experiences with fraud during brainstorming

(PTR_EXPERIENCES, p = 0.01), which is a specific indicator of treating the session as a

training opportunity. Partners in the treatment condition more often addressed issues of

effectiveness and efficiency to promote an appropriately calibrated response to fraud risk

(CALIBRATED_RESPONSE, p = 0.01). Additionally, partners in the treatment condition more

often discussed the importance of professional skepticism with respect to specific accounts on

the engagement with a higher level of fraud risk (PS_SPECIFIC, p = 0.02). The pattern of these

univariate comparisons illustrates that the intervention given to partners in the treatment

condition positively shifted managers’ and seniors perceptions of topics discussed and issues

emphasized by the audit partner during brainstorming, thereby providing evidence that the

intervention successfully manipulated these partner behaviors.

20

In contrast, there are no significant univariate differences between the other three

indicators of partner behavior, TRAINING_OPP, EFFECTIVE_EFFICIENT, or PS_GENERAL.

The treatment (control) condition means are as follows: TRAINING_OPP, means = 6.57 (6.44),

EFFECTIVE_EFFICIENT means = 7.15 (6.94), and PS_GENERAL, means = 100 percent (96

percent). The relatively high levels (above the midpoints and nearly 100 percent) of the

TRAINING_OPP, EFFECTIVE_EFFICIENT, and PS_GENERAL measures, together with the

lack of differences in these measures between conditions, implies that audit partners address

these relatively more general discussion topics and issues of emphasis routinely in practice.

There are likely upper-bound limits on the emphasis that partners can reasonably place on these

considerations simultaneously in practice; therefore, it is not entirely surprising that we find no

differences in these measures across conditions. Moreover, our experimental intervention

successfully manipulated the behavior of audit partners on more specific dimensions where

univariate tests indicate larger improvement opportunities in practice.

These results help to understand mixed findings in prior research regarding the efficacy

of interventions that remind auditors about professional skepticism in general (i.e., Carpenter and

Reimers 2013; Gissel 2013). These studies use an intervention that is consistent with the wording

in AU 316, that is, to maintain “the proper state of mind throughout the audit” (paragraph 14),

which emphasizes professional skepticism in general. Our results indicate that an intervention

emphasizing professional skepticism with respect to specific accounts on the engagement with a

higher level of fraud risk will be more effective than an intervention requesting partners to

emphasize professional skepticism in general, perhaps reflecting the fact that auditors already

receive significant general communications about professional skepticism in practice.

21

Measures for Process and Outcome Effects

Dependent Variables

We test the association between the intervention and both process and outcome measures

of fraud brainstorming. Process variables include PS_CHG_GENERAL, PS_CHG_SPECIFIC,

DISCUSSION_MGT, and DISCUSSION_RESP, and SESSION_LENGTH. PS_CHG_GENERAL

and PS_CHG_SPECIFIC measure changes in the self-assessed levels of professional skepticism

on the engagement during fraud brainstorming.14

We calculate these variables as the difference

between self-assessed levels of professional skepticism after brainstorming and self-assessed

levels of professional skepticism before brainstorming, where participants self-assess their own

levels of professional skepticism on the sample engagement relative to an “average” or “normal”

client on a scale from 1 (much lower than normal) to 10 (much higher than normal).15

PS_CHG_GENERAL measures the change in the individual’s self-assessed level of professional

skepticism on the engagement in general. PS_CHG_SPECIFIC measures the change in the

individual’s self-assessed level of professional skepticism with respect to specific accounts on

the engagement with a higher level of fraud risk. DISCUSSION_MGT is the extent of discussion

during brainstorming about how management might perpetrate fraud, on a scale from 1 (very

low) to 10 (very high). DISCUSSION_RESP is the extent of discussion during brainstorming

about audit responses to fraud risk, on a scale from 1 (very low) to 10 (very high).

SESSION_LENGTH is the number of minutes spent brainstorming. H1 predicts these process

variables will be positively associated with TREATMENT.

14

We do not measure trait skepticism (e.g., Hurtt 2010) because we do not expect the intervention to influence traits

of managers and seniors. 15

Participants self-assess each of these levels of professional skepticism using the survey instrument they receive

after the fraud brainstorming session is complete.

22

Fraud risk factor identification outcome variables include RISKS_NUMBER,

RISKS_NEW, %REV_REC, and %OVERRIDE. RISKS_NUMBER is the number of fraud risks

identified during fraud brainstorming. RISKS_NEW is the number of fraud risks identified during

brainstorming that are new for the current year audit. We measure both RISKS_NUMBER and

RISKS_NEW using quantitative counts of risk factor data. H2 predicts that RISKS_NUMBER and

RISKS_NEW will be positively associated with TREATMENT.

The other two fraud risk factor identification outcome variables, %REV_REC and

%OVERRIDE, measure qualitative characteristics of risk factors. %REV_REC is the percentage

of fraud risk factors identified that relate to revenue recognition. %OVERRIDE is the percentage

of fraud risk factors identified that relate to management override of controls. AU 316 states

“auditors should ordinarily presume” that improper revenue recognition is a fraud risk (AICPA

2002b, 41). Additionally, AU 316 directs auditors to “consider the possibility that management

override of controls could occur” in the context of fraud risk (AICPA 2002b, 42). We analyze

%REV_REC and %OVERRIDE to determine whether TREATMENT is associated with the types

of fraud risk factors identified, and we make no directional predictions related to these

associations. Table 3 provides examples of fraud risk factors coded as relating to revenue

recognition and management override of controls.

INSERT TABLE 3 HERE

Fraud risk response outcome variables include PROC_NUMBER, PROC_NEW,

PROC_UNPRED, TAILOR, %RELATE_NATURE, %RELATE_EXTENT, and

%RELATE_TIMING. PROC_NUMBER is the number of procedures planned to respond to fraud

risks in the current year audit. PROC_NEW is the number of planned procedures that are new for

the current year audit. PROC_UNPRED is the number of planned procedures intended to

23

incorporate an element of unpredictability in the audit. TAILOR is a dichotomous variable equal

to one if the engagement team eliminated fraud risk responses that had been used in prior year

audits (i.e., tailoring the audit plan to the current year audit); equals zero otherwise. H2 predicts

that PROC_NUMBER, PROC_NEW, PROC_UNPRED, and TAILOR will be positively

associated with TREATMENT.

The other three fraud risk response outcome variables, %RELATE_NATURE,

%RELATE_EXTENT, and %RELATE_TIMING, measure qualitative characteristics of fraud risk

responses. Each of these variables is calculated as the percentage of responses to a fraud risk

factor that relate to the nature, extent, and timing, respectively, of planned procedures. We

analyze these variables to determine whether TREATMENT is associated with the types of

planned responses to fraud risk, and we make no directional predictions related to these

associations. Table 3 provides examples of these fraud risk response variables.

Control Variables

We include in hypothesis-testing models control variables used in prior research (Brazel

et al. 2010). CLIENT_SIZE is based on revenue and is coded as follows: 1 = < $100 million, 2 =

$100 million - $500 million, 3 = > $500 million - $1 billion, 4 = > $1 billion - $5 billion, 5 = >

$5 billion. PUBLIC is a dichotomous variable equal to one if the client is publicly traded; equals

zero otherwise. INDUSTRY_FS is a dichotomous variable equal to one if the client is in the

financial services industry; equals zero otherwise. INDUSTRY_GV/NP is a dichotomous variable

equal to one if the client is in the government or not-for-profit industry; equals zero otherwise.

INDUSTRY_MFG is a dichotomous variable equal to one if the client is in the manufacturing

24

industry; equals zero otherwise.16

INHERENT_RISK is the level of inherent risk associated with

the overall engagement on a scale from 1 (low) to 10 (high). FRAUD_RISK is the level of fraud

risk associated with the overall engagement on a scale from 1 (low) to 10 (high). As in Brazel et

al. (2010), we control for COMPLEXITY, which is measured as the number of auditors assigned

to the engagement (coded as follows: 1 = 0-5 auditors, 2 = 6-10 auditors, 3 = 11-15 auditors, 4 =

16-20 auditors, and 5 = > 20 auditors) divided by CLIENT_SIZE. TEAM_EXPERTISE is the

engagement team’s level of expertise on the client on a scale from 1 (very low) to 10 (very high).

PTR_CLIENT_EXPC is the level of the experience of the engagement partner on the respective

engagement, coded as 1 = first year on engagement, 2 = relatively new to engagement, and 3 =

moderate or significant amount of experience on engagement.17

MGR_CLIENT_EXPC is the

number of months the lead engagement manager has served on the engagement.

SR_CLIENT_EXPC is the number of months the lead engagement senior has served on the

engagement. EXPERIENCE_FRR is the number of engagements the respondent served on in

which fraudulent financial reporting was identified, coded as 1 = 0, 2 = 1-2, 3 = >2. Experience

and expertise are elements of fraud knowledge, which Hammersley (2011) notes is a critical

determinant of fraud hypothesis generation. MANAGER1 is a dichotomous variable equal to one

if the survey respondent is a manager; zero if the survey respondent is a senior.

FORMAL_FORMAT is a dichotomous variable equal to one if the nature of the format of the

discussion for fraud brainstorming is either round robin or nominal group; equals zero

16

INDUSTRY_MISC is a dichotomous variable equal to one if the client's industry is Retail, Energy, High

Tech/Communications, Healthcare/Pharmaceuticals, or Other; zero otherwise. We exclude this variable from the

analyses to avoid singularities in the models. 17

This variable implicitly controls for partner turnover. We calculate it as a categorical variable because the

manager and senior participants likely do not know the exact number of months that the audit partner served on the

engagement.

25

otherwise.18

We make non-directional predictions for all control variables, except for the

following variables in the models with process-related dependent variables. In those models, we

expect that CLIENT_SIZE, PUBLIC, INHERENT_RISK, FRAUD_RISK, COMPLEXITY, and

EXPERIENCE_FFR will be positively associated with the dependent variables.

IV. RESULTS

Descriptive Results

Table 4, Panel A presents descriptive statistics on the dependent variables. With respect to

process-related dependent variables, changes in professional skepticism in general and with

respect to specific accounts, PS_CHG_GENERAL and PS_CHG_SPECIFIC, are higher (p = 0.02

and p = 0.02, respectively) in the treatment condition (mean = 0.73 and mean = 0.87,

respectively) than in the control condition (mean = 0.45 and mean = 0.51, respectively). The

mean of DISCUSSION_MGT is 7.12 on a scale from 1 (very low) to 10 (very high), with no

significant univariate difference between the two conditions. The mean DISCUSSION_RESP is

7.03 on a scale from 1 (very low) to 10 (very high), with a marginally lower extent of discussion

about responses to fraud risk in the treatment condition (p = 0.09). The mean

SESSION_LENGTH is about 35 minutes, and is longer (by about 10 minutes) in the treatment

condition (p = 0.04).

INSERT TABLE 4, PANEL A HERE

With respect to fraud risk factor identification outcome dependent variables,

RISKS_NUMBER is 3.43 with no differences between the conditions. The number of new fraud

risks identified during brainstorming, RISKS_NEW, is relatively low with an overall sample

mean of 0.52. However, RISKS_NEW is higher (p = 0.01) in the treatment condition (mean =

18

We collected data on other control variables used in Brazel et al. (2010), but do not include control variables in

our hypothesis-testing models that are consistently insignificant.

26

0.72) than in the control condition (mean = 0.33). The mean percentage of risk factors related to

revenue recognition, %REV_REC, is 30 percent, with no significant differences between the

conditions. The mean percentage of risk factors related to management override of controls,

%OVERRIDE, is 19 percent, with no significant differences between the conditions.

With respect to fraud risk response outcome dependent variables, PROC_NUMBER, is

5.55, with no significant differences between the conditions. The number of new planned

procedures, PROC_NEW, is low with a mean of 0.42. However, PROC_NEW is higher (p =

0.05) in the treatment condition (mean = 0.58) compared to the control condition (mean = 0.29).

The mean number of procedures planned to incorporate an element of unpredictability in the

audit, PROC_UNPRED, is 1.72, with no significant differences between the conditions. Finally,

engagement teams seem reluctant to eliminate prior-year procedures, TAILOR, with a mean of 20

percent. However, engagement teams are marginally more likely to eliminate procedures in the

control condition than in the treatment condition (25 percent and 14 percent, respectively; p =

0.08). The mean percentage of fraud risk responses relating to the nature of planned procedures,

%RELATE_NATURE, is 86 percent and the mean percentage of fraud risk responses relating to

the extent of planned procedures, %RELATE_EXTENT, is 13 percent; there are no significant

differences between the conditions for either variable. The mean percentage of fraud risk

responses relating to the timing of planned procedures, %RELATE_TIMING, is two percent and

is lower in the treatment condition (p = 0.02) than in the control condition.

Table 4, Panel B presents descriptive statistics on the control variables. CLIENT_SIZE is

in the range of $500 million to $1 billion, and clients in the control condition are marginally

larger (p = 0.05). Sixty-two percent of the sample is PUBLIC, and this percentage is significantly

higher in the control condition (p = 0.02). In terms of industry membership, the largest

27

representations are in financial services (29 percent) and manufacturing (25 percent). We also

control for membership in the government/not-for-profit industry (seven percent). Means of

INHERENT_RISK and FRAUD_RISK are 4.71 and 4.13, respectively on scales from 1 (low) to

10 (high), and do not differ by experimental condition. Mean COMPLEXITY is 0.85, which is

similar to Brazel et al. (2010), and also does not differ by experimental condition.

INSERT TABLE 4, PANEL B HERE

The mean TEAM_EXPERTISE is 7.75, which indicates a perception that sample

engagement teams possess a relatively high level of expertise. The mean PTR_CLIENT_EXPC is

2.63, which indicates roughly moderate levels of experience on the client.19

Neither of these two

variables differs by experimental condition. The means of MGR_CLIENT_EXPC and

SR_CLIENT_EXPC are approximately 37 months and 23 months, respectively.

SR_CLIENT_EXPC is lower in the treatment condition (p = 0.04). The number of engagements

on which the respondent experienced fraudulent financial reporting in the past is quite low, with

a mean of 1.24 (indicating zero to two engagements), with no differences between experimental

conditions. The mean of MANAGER1 is 0.51, indicating that 51% and 49% of the surveys were

completed by managers and seniors, respectively. In terms of audit firm representation, 77

percent of the sample is from Firm A, 16 percent is from Firm B, and 7 percent is from Firm C

(untabulated). With respect to brainstorming session format, open discussion is the predominant

method (88 percent of the time), with round robin (17 percent) and nominal group (5 percent)

being used much less frequently.20, 21

Format type does not differ by experimental condition.

19

Ten respondents indicated the partner was new to the engagement for the current year audit. Inferences are

unchanged when we exclude these observations from the analyses. 20 Format type does not differ significantly by audit firm. 21

At 2.31 or less, the VIFs from the hypothesis-testing model estimated using linear regression are all well below

the 10.00 threshold recommended by Belsley et al. (1980). Thus, collinearity does not appear to be a concern in our

hypothesis-testing analyses.

28

Hypothesis Tests

Process Variables

Table 5 focuses on dependent variables relating to brainstorming processes. 22

There is a

positive association between TREATMENT and both PS_CHG_GENERAL (t = 1.95, p = 0.03)

and PS_CHG_SPECIFIC (t = 1.74, p = 0.04), consistent with expectations in H1. There is also a

positive and marginally significant association between TREATMENT and DISCUSSION_MGT

(t = 1.31, p = 0.10), but not DISCUSSION_RESP (t = -1.23, p = 0.22). The results further show a

positive association between TREATMENT and SESSION_LENGTH (t = 1.84, p = 0.04). This

implies that the amount of audit firm resources applied to implement the field intervention is not

excessive (with a mean increment of about 10 minutes) and provides context for potential budget

management concerns in the field. When considered with the positive outcome effects (described

subsequently), the results for SESSION_LENGTH suggest that fraud brainstorming quality

outcome effects achieved via the intervention come at a relatively low cost. Taken together, these

results imply that the intervention yields larger increases in professional skepticism, both in

general and with respect to specific accounts with a higher risk of fraud, more discussion about

how management might perpetrate fraud, and longer fraud brainstorming sessions, providing

support for H1.

INSERT TABLE 5 HERE

CLIENT_SIZE is positively associated with PS_CHG_SPECIFIC (t = 1.74, p = 0.04),

DISCUSSION_MGT (t = 2.41, p < 0.01), and SESSION_LENGTH (t = 1.97, p = 0.03). PUBLIC

is positively associated with DISCUSSION_MGT (t = 1.68, p = 0.05), but is unexpectedly

22

We cluster standard errors by engagement in all models to control for within-engagement correlation. The field

experiment was conducted in the same manner at all three of the firms participating in the study and we do not

expect within-firm correlation. However, we clustered standard errors by audit firm in all models and find that doing

so does not change the results.

29

negatively associated with PS_CHG_GENERAL or PS_CHG_SPECIFIC (t = -1.77, p = 0.06; t =

-2.52, p = 0.01, respectively), which may indicate that professional skepticism is already quite

high for these engagements. FRAUD_RISK is positively associated with PS_CHG_SPECIFIC (t

= 1.31, p = 0.09), DISCUSSION_MGT (t = 1.93, p = 0.03), and DISCUSSION_RESP (t = 2.95, p

< 0.01). TEAM_EXPERTISE is positively associated with both DISCUSSION_MGT (t = 2.31, p

= 0.02) and DISCUSSION_RESP (t = 2.10, p = 0.04).

We also find interesting associations between brainstorming processes and partner,

manager, and senior experience on the client. PTR_CLIENT_EXPC is negatively and marginally

significantly associated with PS_CHG_GENERAL (t = -1.77, p = 0.08) and DISCUSSION_MGT

(t = -1.96, p = 0.05). These results imply an inertia effect whereby longer audit partner tenure on

an engagement yields “stickiness” in processes associated with brainstorming sessions.

Interestingly, the results imply the opposite with respect to brainstorming processes and the

number of months the lead engagement manager has served on the engagement. Specifically,

MGR_CLIENT_EXPC is positively and marginally significantly associated with

DISCUSSION_MGT (t = 1.84, p = 0.07) and DISCUSSION_RESP (t = 1.93, p = 0.06),

suggesting that continuity at the manager level enhances brainstorming processes.

SR_CLIENT_EXPC is marginally negatively associated with PS_CHG_SPECIFIC (t = -1.67, p =

07) and is negatively associated with DISCUSSION_MGT (t = -2.53, p = 0.03). Finally,

MANAGER1 is negatively associated with both PS_CHG_GENERAL and PS_CHG_SPECIFIC

(t = -3.94, p < 0.01; t = -2.40, p = -0.02, respectively), and is positively associated with

DISCUSSION_MGT (t = 2.89, p < 0.01), suggesting that seniors and managers experience

brainstorming sessions somewhat differently. Finally, a FORMAL_FORMAT (nominal group or

round robin) is positively associated with SESSION_LENGTH (t = 2.21, p = 0.03).

30

Fraud Risk Factor Outcome Variables

Table 6 focuses on fraud risk factor identification outcome dependent variables

associated with brainstorming. With respect to quantitative measures, there is a positive

association between TREATMENT and RISKS_NUMBER (t = 1.75, p = 0.04), and a positive

association between TREATMENT and RISKS_NEW (t = 2.10, p = 0.02), both consistent with

H2. There are no significant associations between TREATMENT and measures of qualitative

characteristics of risk factors (%REV_REC and %OVERRIDE). This implies that the fraud risk

profile of clients in the field does not differ by experimental condition. Moreover, this lack of

associations suggests that the intervention did not promote incremental consideration of

commonly identified fraud risk factors that specifically require attention under AU 316.

INSERT TABLE 6 HERE

Regarding control variables, there is a positive association between INHERENT_RISK

and both RISKS_NUMBER (t = 2.33, p = 0.02) and RISKS_NEW (t = 2.21, p = 0.03).

CLIENT_SIZE is negatively associated with RISKS_NEW (t = -2.15, p = 0.04); since all

engagements in the sample were continuing engagements with the respective audit firms, this

result is consistent with audit firms deploying more resources in prior years to identify fraud

risks at larger clients. Similarly, PTR_CLIENT_EXPC is negatively associated with RISKS_NEW

(t = -2.05, p = 0.04), suggesting engagement teams accumulate relevant fraud risk factors over

time. PTR_CLIENT_EXPC is positively associated with %OVERRIDE (t = 2.93, p < 0.01),

potentially alleviating concerns that audit partners might grow increasingly comfortable with

management as partner tenure increases. Lastly, MANAGER1 is positively and marginally

significantly associated with both %REV_REC (t = 1.81, p = 0.08) and %OVERRIDE (t = 1.72, p

31

= 0.09), suggesting that managers were more cognizant of specific provisions in AU 316 when

completing the surveys than seniors.

Fraud Risk Response Outcome Variables

Table 7 focuses on fraud risk response outcome dependent variables associated with

brainstorming. We find no significant associations between TREATMENT and any of the

measures related to audit procedures: number of procedures (PROC_NUMBER), new procedures

(PROC_NEW), or procedures intended to incorporate an element of unpredictability

(PROC_UNPRED).23

The lack of treatment effects for H2 related to all three procedures

variables is unexpected, but consistent with prior literature that shows auditors often have

difficulty linking evaluations of fraud risk with appropriate fraud risk responses (e.g., Mock and

Turner 2005; Hammersley 2011; Hammersley et al. 2011), or with substantive test modifications

in other audit tasks (Mauldin and Wolfe 2014). Moreover, in contrast with expectations in H2,

the results show a negative association between TREATMENT and TAILOR (z = -3.15, p < 0.01),

which indicates that the intervention is associated with reduced tailoring such that there is a

lower likelihood of eliminating fraud risk responses that had been used in the prior year.24

INSERT TABLE 7 HERE

Conversely, our analyses of qualitative fraud risk response outcome dependent variables

provide interesting insights related to H2. There is a positive and marginally significant

association between TREATMENT and %RELATE_NATURE (t = 1.86, p = 0.07), and negative

associations between TREATMENT and both %RELATE_EXTENT (t = -2.03, p = 0.05) and

23

It is interesting to note that univariate tests show PROC_NEW is greater (t = 1.65, p = 0.05) for engagements in

the treatment condition (mean = 0.58) than for engagements in the control condition (mean = 0.29). 24

We perform a median split on the FRAUD_RISK variable and re-estimate Model 13 (TAILOR) using subsamples

of observations with high fraud risk (greater than or equal to median) and low fraud risk (less than or equal to

median). The association between TREATMENT and TAILOR is negative for both the high fraud risk subsample (t =

-2.50, p = 0.01) and the low fraud risk subsample (t = -3.20, p < 0.01). Therefore, the reluctance to eliminate fraud

risk responses from the prior year audit in the treatment condition appears to occur regardless of the level of fraud

risk.

32

%RELATE_TIMING (t = -2.04, p = 0.05). These results reveal that, compared to engagement

teams in the control condition, engagement teams in the treatment condition respond to fraud risk

through relatively more modifications to the nature of planned procedures (e.g., strengthening

procedures performed, developing and performing new procedures) and relatively fewer

modifications to the extent (e.g., increased sample size, use of lower scope) and timing (e.g.,

testing at final vs. interim, testing at interim in addition to final) of planned procedures. This is

consistent with fraud risk responses of engagement teams in the treatment condition focusing

more on what is being done to address fraud risks and focusing less on how much is being done

and when it is being done. If modifications to the nature of planned procedures address fraud risk

factors more effectively and/or efficiently than modifications to the extent or timing of planned

procedures, then the intervention is associated with favorable fraud brainstorming outcomes.

Interesting findings regarding control variables include a negative association between

FRAUD_RISK and %RELATE_TIMING (t = -2.27, p = 0.03), indicating that auditors are less

likely to respond to fraud risk by modifying the timing of planned procedures when they evaluate

heightened fraud risk. COMPLEXITY is negatively associated with both PROC_NEW (t = -1.69,

p = 0.09) and PROC_UNPRED (t = -2.40, p = 0.02), and is positively associated with both

TAILOR (t = 1.67, p = 0.09) and %RELATE_EXTENT (t = 2.15, p = 0.04). These results

highlight the difficulty of responding to fraud risks on complex audit engagements in that

auditors struggle to identify new procedures to respond to fraud risk, struggle to incorporate

unpredictability into these audits, but that auditors respond to this complexity by carefully

tailoring and planning larger sample sizes. TEAM_EXPERTISE is positively and marginally

significantly associated with PROC_UNPRED (t = 1.94, p = 0.06), suggesting client-specific

team expertise helps auditors to incorporate elements of unpredictability into the audit plan.

33

PTR_CLIENT_EXPC, is negatively and marginally significantly associated with TAILOR (t = -

1.87, p = 0.06), suggesting further potentially negative consequences of audit partner tenure with

respect to fraud brainstorming outcomes (i.e., “stickiness”) and complementing the findings

related to PRT_CLIENT_EXPC in the models analyzing brainstorming processes (discussed

previously). MANAGER1 is positively and marginally significantly associated with

%RELATE_NATURE (t = 1.71, p = 0.09) and negatively and marginally significantly associated

with %RELATE_EXTENT (t = -1.67, p < 0.10), suggesting that, compared to seniors, managers

focused more on what was being done to respond to fraud risk and less on how much was being

done to respond to fraud risk when completing the surveys. These results complement those in

Hammersley et al. (2011), which reveal that audit seniors often respond to fraud risk through

indiscriminate sample size increases rather than through effective audit program modifications.

Finally, the results show a negative and marginally significant association between

FORMAL_FORMAT and TAILOR (t = -1.67, p < 0.10), implying that such a format yields less

willingness to tailor via eliminating previously used procedures.

Supplemental Analysis

We analyze the association between TAILOR and TREATMENT separately for public and

private clients to examine the possibility that the unwillingness to tailor relates to concerns over

justifying the elimination of fraud risk responses during PCAOB inspections. TREATMENT is

negatively associated with TAILOR in Model 13 when we include only public clients in the

model (t = -2.18, two-tailed p = 0.03). Additionally, a two-sample t-test indicates a significant

univariate difference between the mean values of TAILOR for public clients in the treatment and

control conditions (t = -2.11, p = 0.04). Due to a lack of variation between TAILOR and several

independent variables in the sample of private clients, we lose a number of observations and are

34

unable to fit Model 13 to the sample of private clients. However, a two-sample t-test indicates no

significant univariate difference between the mean values of TAILOR for private clients in the

treatment versus control conditions (n = 71, t = .39, one-tailed p = .35), suggesting that public

client engagements drive the unexpected negative association between TREATMENT and

TAILOR. This is consistent with engagement teams being concerned about PCAOB inspections,

which encourages them to avoid eliminating procedures performed in prior years.

V. CONCLUSION

We conduct a field experiment to test whether an intervention can influence the approach

audit partners take in leading fraud brainstorming sessions and whether this is associated with

brainstorming processes and outcomes in natural hierarchical audit teams. The results suggest

partners tend to address relatively more general discussion topics and issues of emphasis relevant

to brainstorming routinely in practice (emphasizing the session as a training opportunity in

general, discussing effective/efficient brainstorming in general, and discussing professional

skepticism in general.) The intervention successfully improved the approach of partners with

respect to more specific discussion topics and issues of emphasis relevant to brainstorming

where there appears to be room for improvement in practice (discussion of prior experiences

with fraud during brainstorming, discussion of issues of effectiveness/efficiency to promote an

appropriately calibrated response to fraud risk, and discussion of the importance of professional

skepticism with respect to specific accounts on the engagement with a higher level of fraud risk).

Our results extend prior research in interesting ways. For example, while Lynch et al.

(2009) find that a content facilitation intervention improves the identification of relevant fraud

risk factors using student subjects and Carpenter and Reimers (2013) find a positive association

between the degree of partner emphasis on professional skepticism and fraud risk assessments

35

using managers, Gissel (2013) finds no association between manager emphasis on professional

skepticism and the sharing of private information using staff and seniors. Our results reconcile

and extend these prior findings by showing that the effectiveness of an intervention in fraud

brainstorming depends on whether the intervention is specific (as opposed to general) and is

related to audit partner leadership behavior where there is room for improvement in practice.

Consistent with our expectations related to fraud brainstorming processes, the

intervention is associated with increases in professional skepticism during brainstorming, both in

general and with respect to specific accounts with a higher level of fraud risk, more discussion

about how management might commit fraud, and longer brainstorming sessions. Our process-

related results extend Carpenter’s (2004) findings related to professional skepticism and are

consistent with her finding that when a partner expresses a preference for effectiveness versus

efficiency the audit team spends more time in brainstorming.

Consistent with our expectations related to fraud brainstorming outcomes, the

intervention is associated with the identification of a higher number of fraud risks and a higher

number of new fraud risks. However, we find no association between the intervention and the

percentages of fraud risk factors identified that relate to revenue recognition and relate to

management override of controls, suggesting that engagement teams in both experimental

conditions encounter similar conditions in the field. In contrast with our expectations, we find no

evidence of associations between the intervention and the generation of audit responses to fraud

risks and a negative association between the intervention and the willingness of engagement

teams to tailor the audit plan to the current year audit by eliminating fraud risk factors identified

in prior year audits. This latter result is attributable to the public clients in the sample. We do

find associations between the intervention and qualitative characteristics of responses to fraud

36

risk that are consistent with engagement teams in the treatment condition focusing more on what

to do to respond to fraud risk, and less on how much to do or when to do it.

Taken together, our results extend the fraud brainstorming literature that examines the

efficacy of explicit instructions. Therefore, our study also has important implications for

practitioners and regulators. We designed the intervention with the help of senior leadership at

the audit firms that participated in this study; these individuals were interested in improving

auditor judgments in brainstorming that require professional skepticism. These instruction

guidelines are simple, actionable items that audit firms can incorporate into methodology and

training programs to enhance brainstorming effectiveness and address recent PCAOB concerns

about “mechanical implementation” of AU 316 (PCAOB 2007, 4). To this end, our study

illustrates that an intervention can be an effective means to re-focus audit teams on the

importance of fraud detection and deterrence and promote diligent compliance with AU 316.

Limitations and Future Research

One limitation of this study is that it seems audit partners already address relatively more

general discussion topics and issues of emphasis relevant to fraud brainstorming routinely in

practice (i.e., emphasizing the session as a training opportunity in general, discussing

effective/efficient brainstorming in general, and discussing professional skepticism in general).

This implies high quality professional behavior for which no intervention is necessary to

improve brainstorming processes and outcomes. Future research might explore other potential

actionable interventions in this task setting, as well as investigating interventions that may be

applicable to engagement team members other than the audit partner. Future research can also

examine the effect of multi-period interventions. Our study tests the effectiveness of an

37

intervention in a single brainstorming session and it is possible that the effectiveness of the

intervention will change in subsequent brainstorming sessions.

A second limitation relates to the fact that due to client confidentiality constraints, we

were unable to oversee the actual selection and assignment of engagements in the study. As

previously noted, we took steps to emphasize the importance of random selection and random

assignment during our meetings with the senior audit partners that served as the “local office

champions” of the research study. We have no reason to believe that any systematic bias exists in

the selection of engagements into the sample or the assignment of engagements to experimental

conditions. It is possible that the senior audit partners might have chosen to include only “good”

clients in the sample; however, the descriptive statistics reveal variation in measures of inherent

risk (min/max of 1-10) and fraud risk (min/max of 1-9) in the overall sample, which suggests

that this is not the case. It is difficult to predict whether and how the senior audit partners might

have been motivated to systematically bias assignment of engagements to experimental

conditions. However, we find the same levels of variation in measures of inherent risk (min/max

of 1-10) and fraud risk (min/max of 1-9) in both the treatment and control conditions, which

alleviates concerns about potential biases in the assignment “good” clients to a particular

experimental condition.

Finally, because of institutional review requirements, the partners all knew they were

participating in research relating to their fraud brainstorming session. This knowledge might

somehow have affected their behavior, e.g., by being more diligent than normal. Random

assignment to experimental conditions should alleviate this concern. Further, it is important to

note that the managers and seniors did not know about the experiment while they participated in

the brainstorming session, and it is their perceptions upon which we base our measures.

38

REFERENCES

American Institute of Certified Public Accountants (AICPA). 2002a. Consideration of Fraud in

a Financial Statement Audit. Statements on Auditing Standards No. 99. New York, NY:

AICPA.

American Institute of Certified Public Accountants (AICPA). 2002b. Consideration of Fraud in

a Financial Statement Audit. AU Section 316. New York, NY: AICPA.

Asare, S. K., and A. M. Wright. 2004. The effectiveness of alternative risk assessment and audit

program planning tools in a fraud setting. Contemporary Accounting Research 21 (2):

325-352.

Bedard, J.C., and K. M. Johnstone. 2010. Audit partner tenure and audit planning and pricing.

Auditing: A Journal of Practice & Theory 29 (2): 45-70.

Bellovary, J. L., and K. M. Johnstone. 2007. Descriptive evidence from audit practice on SAS

No. 99 brainstorming activities. Current Issues in Auditing 1 (1): 1–11.

Belsley, D. A., E. Kuh, and R. E. Welsch. 1980. Regression Diagnostics Identifying Influential

Data and Sources of Collinearity. John Wiley & Sons: New York.

Brazel, J. F., T. D. Carpenter, and J. G. Jenkins. 2010. Auditors’ use of brainstorming in

consideration of fraud: Reports from the field. The Accounting Review 85 (4): 1273–

1301.

Carmichael, D. R. 2004. The PCAOB and the social responsibility of the independent auditor.

Accounting Horizons 18 (2): 127–133.

Carpenter, T. D. 2004. Partner Influence, Team Brainstorming, and Fraud Risk Assessment:

Some Implications of SAS No. 99. Unpublished dissertation, Florida State University.

Carpenter, T. D. 2007. Audit team brainstorming, fraud risk identification, and fraud risk

assessment: Implications of SAS No. 99. The Accounting Review 82 (5): 1119–1140.

Carpenter, T. D., J. L. Reimers, and P. Z. Fretzwell. 2011. Internal Auditors’ Fraud Judgments:

The Benefits of Brainstorming in Croups. Auditing: A Journal of Practice & Theory 30

(3): 211–224.

Carpenter, T. D., and J. L. Reimers. 2013. Professional skepticism: The effects of a partner’s

influence and the level of fraud indicators on auditors’ fraud judgments and actions.

Behavioral Research in Accounting 25 (2): 45–69.

Chen, X. C., K. T. Trotman, and F. H. Zhou. 2013. Electronic fraud brainstorming in

hierarchical audit teams: Does interaction help or hurt? Working paper, University of

Illinois at Urbana-Champaign and University of New South Wales.

Committee of Sponsoring Organizations of the Treadway Commission (COSO). 2011. Internal

Control – Integrated Framework. Available at http://www.coso.org/ic-

integratedframework-summary.htm.

Gissel, J. L. 2011. Private Information Sharing during SAS 99 Brainstorming: Effects of

Psychological Safety and Professional Skepticism. Working Paper, Marquette University.

Hammersley, J. S. 2011. A review and model of auditor judgments in fraud-related planning

tasks. Auditing: A Journal of Practice & Theory 30 (4): 101–128.

Hammersley, J. S., K. M. Johnstone, and K. Kadous. 2011. How do audit seniors respond to

heightened fraud risk? Auditing: A Journal of Practice & Theory 30 (3): 81–101.

Hoffman, V. B., and M. F. Zimbelman. 2009. Do strategic reasoning and brainstorming help

auditors change their standard audit procedures in response to fraud risk? The Accounting

Review 84 (3): 811–837.

Hurtt, R. K. 2010. Development of a Scale to Measure Professional Skepticism. Auditing: A

Journal of Practice & Theory 29 (1): 149–171.

39

Kerr, N. L., and R. S. Tindale. 2004. Group performance and decision making. Annual Review of

Psychology 55 (1): 623–655.

Klein, G. 1999. Sources of Power: How People Make Decisions. Cambridge, MA: MIT Press.

Landis, M., S. I. Jerris, and M. Braswell. 2008. Better brainstorming. Journal of Accountancy

206 (4): 70–73.

Lynch, A. L., U. S. Murthy, and T. J. Engle. 2009. Fraud brainstorming using computer-

mediated communication: The effects of brainstorming technique and facilitation. The

Accounting Review 84 (4): 1209–1232.

Mauldin, E. G., and C. J. Wolfe. 2014. How do auditors address control deficiencies that bias

accounting estimates? Contemporary Accounting Research (forthcoming).

Mock, T. J., and J. L. Turner. 2005. Auditor identification of fraud risk factors and their impact

on audit programs. International Journal of Auditing 9 (1): 59–77.

Osborn, A. F. 1957. Applied Imagination: Principles and Procedures of Creative Thinking. New

York: Charles Scribner’s Sons.

Public Company Accounting Oversight Board (PCAOB). 2007. Observations on Auditors’

Implementation of PCAOB Standards Relating to Auditors’ Responsibilities with Respect

to Fraud. Release No. 2001-001, January 22. Washington, DC: PCAOB. Available at

http://pcaobus.org/Inspections/Documents/2007_01-22_Release_2007-001.pdf.

Swieringa, R. J., and K. E. Weick. 1982. An assessment of laboratory experiments in accounting.

Journal of Accounting Research 20 (Supplement): 56-101.

Trotman, K. T., R. Simnett, and A. Khalifa. 2009. Impact of the type of audit team discussions

on auditors’ generation of material frauds. Contemporary Accounting Research 26 (4):

1115–1142.

Wood, D. A., and J. Pickerd. 2011. Problems to avoid when brainstorming fraud risks. The CPA

Journal (April): 64–65.

40

Figure 1

Wording of Experimental Intervention and Research Design/Project Logistics

Panel A. Wording of Partner Memo

[TREATMENT AND CONTROL CONDITIONS] Our firm is collaborating with Professor X at the

University of X on a research study in which you are being asked to participate. The purpose of the

study is to assist the firm in developing its audit methodology and training programs related to fraud

brainstorming. Your participation relates to your role as the engagement partner on the audit client

noted above. Your involvement will be limited to simply completing the annual fraud brainstorming

activities and notifying the contact person in your office when those activities are complete. Your

participation in this study is completely voluntary, and you may opt out of this study by informing the

contact person in your local office that you do not wish to participate. Your participation will be

confidential.

[TREATMENT AND CONTROL CONDITIONS] There is no requirement that the brainstorming be

conducted as a stand-alone meeting; it can be conducted during part of another meeting (e.g., the annual

planning meeting). For research validity purposes, please do not discuss this research with your

colleagues.

[TREATMENT CONDITION ONLY] We ask that you attend to the following five instructions during

fraud brainstorming for this engagement. We provide examples to illustrate possible ways for you to

implement these instructions; however, these examples are not an exhaustive list of implementation

ideas.

1. Emphasize fraud brainstorming as a training/professional development opportunity for the audit

team members present.

Discuss any relevant personal experience on engagements involving fraud.

Discuss some fraud/forensic topics that are relevant to this engagement (e.g., the fraud

triangle with respect to specific accounts with a higher level of fraud risk).

Actively mentor both the audit manager and in-charge auditor in terms of how to most

effectively identify and appropriately respond to fraud risks.

Be cognizant of the fact that your leadership during the session sets the tone for the

engagement team members as they work to appropriately assess and respond to fraud

risk during planning and conduct of the engagement.

2. Discuss the importance of effective and efficient fraud brainstorming.

Discuss the downside risks to the firm of a failure to identify fraud risks early in the

audit.

Discuss features of SAS 99 relevant to this particular engagement, and emphasize the

resources the firm has developed to facilitate fraud brainstorming. Discuss the

importance of documenting compliance with SAS 99 for litigation and review purposes.

Discuss how devoting resources to inappropriate fraud risk procedures can hinder audit

efficiency.

41

3. Discuss the importance of professional skepticism targeted at specific accounts with a

potentially higher level of fraud risk. Emphasize both effectiveness and efficiency to promote an

appropriately calibrated response to fraud risk.

Discuss specific accounts that the engagement team has identified as having a

potentially higher level of fraud risk.

Direct the engagement team to consider whether accounts that have historically been

identified as having a higher level of fraud risk are still appropriately identified as such.

Specifically consider whether a higher assessed level of fraud risk on such accounts is

warranted for the current year audit.

Challenge the engagement team to consider new procedures and/or improvements to

existing procedures.

Discuss relevant and appropriate procedures that can incorporate an element of

unpredictability into the audit.

Compare the client’s financial condition and performance to that of other companies in

its industry using pertinent industry averages (e.g. ratios, trends). Discuss how the

client’s results are either inconsistent or consistent with these averages.

4. Discuss the importance of professional skepticism in general throughout the audit.

Discuss the fact that while certain accounts have a potentially higher level of fraud risk,

there could be fraud in an area not traditionally associated with higher fraud risk.

Encourage the audit manager and in-charge auditor to be alert for anything unusual or

unexpected during the conduct of the audit.

Discuss the importance of considering the risk of management override of internal

controls.

Consider asking the following types of questions to encourage professional skepticism:

(1) What potential frauds may have occurred? (2) How could management conceal the

potential frauds? (3) How could the audit plan be modified to detect concealed fraud?

5. Do not communicate to the engagement team, or any of your other colleagues, that you have

been given these instructions. Doing so will compromise the validity of the research.

[TREATMENT AND CONTROL CONDITION] Federal law requires that we notify you that there are

no risks associated with your participation in this research study, nor are there specific benefits to you

personally. Your participation will help the firm develop its audit methodology and training programs

related to fraud brainstorming. Please notify the contact person in your office when you have completed

the fraud brainstorming activities. At that point, the audit manager and in-charge auditor will complete

a survey about the fraud brainstorming activities/discussion. Your notification of the contact person in

your local office that the fraud brainstorming activities are complete indicates your consent to

participate in this study.

42

Panel B. Research Design/Project Logistics

Local office

contacts

Audit Partners

in Treatment

Group

Audit Partners

in Control

Group

Audit

Manager and

Audit Senior

participants

Completed

Survey

Instruments

Review of surveys

at Audit Firm Level

to Ensure No

Mention of Client-

Identifying

Information

Local office

contacts

Completed

and Audit

Firm-

Reviewed

Survey

Instruments

Memo without

Experimental

Intervention

Memo with

Experimental

Intervention

Experimental

MaterialsResearchers

Notification

that fraud

brainstorming

session is

complete

Uniform

Survey

Instrument Local

Office

Contacts

Completed

Survey

Instruments

43

Appendix A

Variable Definitions

Variable Name Description

Dependent Variables:

PS_CHG_GENERAL

The change in reported professional skepticism on the engagement in general during fraud

brainstorming, calculated using before and after measures on a scale from 1 = much lower

than normal to 10 = much higher than normal.

PS_CHG_SPECIFIC

The change in reported professional skepticism with respect to specific accounts with a

higher level of fraud risk during fraud brainstorming, calculated using before and after

measures on a scale from 1 = much lower than normal to 10 = much higher than normal.

DISCUSSION_MGT The extent of discussion during fraud brainstorming about how management might

perpetrate fraud on a scale from 1 = very low to 10 = very high.

DISCUSSION_RESP The extent of discussion during fraud brainstorming about audit responses to fraud risk on a

scale from 1 = very low to 10 = very high.

SESSION_LENGTH The number of minutes spent in fraud brainstorming.

RISKS_NUMBER Number of fraud risks identified by the engagement team in the current year audit.

RISKS_NEW Number of fraud risks identified by the engagement team for the current year audit, but not

identified in prior year audits.

%REV_REC Percentage of fraud risk factors identified that relate to revenue recognition.

%OVERRIDE Percentage of fraud risk factors identified that relate to management override of controls.

PROC_NUMBER Number of procedures planned by the engagement team to respond to fraud risks in the

current year audit.

PROC_NEW Number of procedures planned by the engagement team to respond to fraud risks in the

current year audit, but not performed in prior year audits.

PROC_UNPRED Number of procedures planned by the engagement team intended incorporate an element of

unpredictability in the current year audit.

TAILOR

A dichotomous variable equal to one if the engagement team eliminated fraud risk

responses that had been used in prior year audits as a result of fraud brainstorming; zero

otherwise.

%RELATE_NATURE Percentage of responses to a fraud risk factor that relate to the nature of planned

procedures.

%RELATE_EXTENT Percentage of responses to a fraud risk factor that relate to the extent of planned

procedures.

%RELATE_TIMING Percentage of responses to a fraud risk factor that relate to the timing of planned

procedures.

Treatment Variable:

TREATMENT A dichotomous variable equal to one if the partner on the engagement received the memo

with the experimental intervention; zero otherwise.

44

Independent Variables:

CLIENT_SIZE

Size of the client based on total revenue, coded as follows: 1 = < $100 million, 2 = $100

million - $500 million, 3 = > $500 million - $1 billion, 4 = > $1 billion - $5 billion, 5 = >

$5 billion.

PUBLIC A dichotomous variable equal to one if the client is publicly traded; zero if the client is

privately owned.

INDUSTRY [FS, GV/NP,

MFG]

Dichotomous variables equal to one if the client's industry is [Financial Services,

Government/Not-for-Profit, or Manufacturing]; zero otherwise.

INDUSTRY_MISC A dichotomous variable equal to one if the client's industry is Retail, Energy, High

Tech/Communications, Healthcare/Pharmaceuticals, or Other; zero otherwise.

INHERENT_RISK Overall engagement-level inherent risk assessment on a scale from 1 = low risk to 10 =

high risk.

FRAUD_RISK Overall engagement-level fraud risk assessment on a scale from 1 = low risk to 10 = high

risk.

COMPLEXITY TEAM SIZE/CLIENT SIZE

TEAM_EXPERTISE The engagement team's level of expertise on the client on a scale from 1 = very low to 10 =

very high.

PTR_CLIENT_EXPC

The experience level of the engagement partner on the respective engagement, coded as 1 =

first year on engagement, 2 = relatively new to engagement, and 3 = moderate or

significant amount of experience on engagement.

MGR_CLIENT_EXPC Number of months the lead engagement manager has served on the engagement.

SR_CLIENT_EXPC Number of months the lead engagement senior has served on the engagement.

EXPERIENCE_FFR The number of engagements the respondent served on in which fraudulent financial

reporting was identified, coded as follows: 1 = 0, 2 = 1-2, 3 = > 2.

MANAGER1 A dichotomous variable equal to one if the survey respondent is a manager; zero if the

survey respondent is a senior.

FORMAL_FORMAT A dichotomous variable equal to one if the nature of the format of the discussion for fraud

brainstorming is round robin or nominal group; zero otherwise

Manipulation Check Variables

TRAINING_OPP

TRAINING_OPP measures the extent to which the audit partner emphasized fraud

brainstorming as a training/professional development opportunity on a scale from 1 (low

emphasis) to 10 (high emphasis).

PTR_EXPERIENCES PTR_EXPERIENCES is a dichotomous variable equal to one if the audit partner discussed

his/her prior experiences with fraud during brainstorming.

EFFECTIVE_

EFFICIENT

EFFECTIVE_EFFICIENT measures the extent to which the audit partner discussed the

importance of effective and efficient brainstorming on a scale from 1 (no discussion) to 10

(significant discussion).

CALIBRATED_RESP

CALIBRATED_RESPONSE is a dichotomous variable equal to one if the audit partner

addressed the issues of effectiveness and efficiency to promote an appropriately calibrated

response to fraud risk; equals zero otherwise.

PS_GENERAL

PS_SPECIFIC is a dichotomous variable equal to one if the audit partner discussed the

importance of professional skepticism with respect to specific accounts on the engagement

with a higher level of fraud risk; equals zero otherwise.

PS_SPECIFIC

PS_GENERAL is a dichotomous variable equal to one if the audit partner discussed the

importance of professional skepticism in general throughout the audit; equals zero

otherwise.

45

TABLE 1

Summary of Previous Brainstorming Research

Panel A: Experimental Designs and Incremental Contribution

Experimental Manipulations in a

Laboratory Setting

Experimental Manipulations in a

Field Setting

Guidance

No mention of the

term

“brainstorming”

Trotman et al. (2009)

Hoffman and Zimbelman (2009)

Brainstorming

without explicit

guidelines

Carpenter (2007)

Hoffman and Zimbelman (2009)

Lynch et al. (2009)

Carpenter et al. (2011)

Chen et al. (2013)

Present Study

Auditing-Specific Laboratory Interventions Intended to Facilitate Fraud Brainstorming:

Brainstorming with

explicit guidelines

Trotman et al. (2009)

Present Study

Content facilitation

Lynch et al. (2009)

Psychological

safety

Gissel (2013)

Partner/manager

emphasis

Effectiveness/

efficiency

Professional

skepticism

Carpenter (2004)

Carpenter and Reimers (2013)

Gissel (2013)

Present study

Present Study

Strategic reasoning Hoffman and Zimbelman (2009)

46

TABLE 1

Summary of Previous Brainstorming Research

Panel B: Dependent Measures

Research Study Process-related

Dependent

Variables

Outcome-related Dependent Variables

Carpenter (2004) Time spent

by audit

teams

Accuracy of fraud risk assessments

Quantity of ideas generated

Professional skepticism

Carpenter (2007) Quantity of ideas generated

Quality of ideas generated

Fraud risk assessments

Carpenter et al.

(2011)

Fraud risk assessments

Quantity and quality of fraud risks

Hoffman and

Zimbelman

(2009)

Modification of nature and extent of planned audit procedures

Budgeted hours

Trotman et al.

(2009)

Quantity of potential misstatements due to fraud

Quality of potential misstatements due to fraud (expert-identified

frauds and expert-identified misstatements)

Proportion of rare potential frauds identified to total potential

frauds identified

Lynch et al.

(2009)

Fraud risk assessments

Quantity of relevant fraud risks

Gissel (2013)

Fraud risk assessments

Willingness to share private information about the fraud

Chen et al.

(2013)

Fraud risk assessments

Quantity of fraud risk factors

Quantity and quality of fraud hypotheses

Auditors’ mental representations for hypothesis testing

Carpenter and

Reimers (2013)

Fraud risk assessments

Quantity of relevant fraud risk factors identified

Quantity of relevant audit procedures

47

TABLE 2

Comparison of Audit Partner Behavior by Experimental Condition

Overall

[n=148]

Treatment

[n=71]

Control

[n=77]

Statb

p-valueb Item

a

n Mean

Std.

Dev. Med

n Mean

Std.

Dev. Med

n Mean

Std.

Dev. Med

TRAINING_OPP

148 6.50 2.47 7.00

71 6.57 2.50 6.00

77 6.44 2.45 7.00

0.32

0.38

PTR_EXPERIENCES

147 0.50 0.50 0.00

70 0.60 0.49 1.00

77 0.40 0.49 0.00

4.95

0.01

EFFECTIVE_EFFICIENT 148 7.04 2.39 7.00

71 7.15 2.42 8.00

77 6.94 2.37 7.00

0.56

0.29

CALIBRATED_RESP

146 0.87 0.34 1.00

70 0.94 0.23 1.00

76 0.80 0.40 1.00

5.15

0.01

PS_GENERAL

147 0.98 0.14 1.00

70 1.00 0.00 1.00

77 0.96 0.19 1.00

1.18

0.14

PS_SPECIFIC

147 0.96 0.20 1.00

70 1.00 0.00 1.00

77 0.92 0.27 1.00

3.87

0.02

a See definitions in Appendix A.

b Continuous variable statistics are t-statistics from means tests. Indicator variable statistics are Chi-Squared statistics with Yates' continuity corrections. All p-values

are from one-tailed tests.

48

TABLE 3

Examples of Qualitative Data Coding

Fraud Risk Factors Fraud Risk Responses

Related to Revenue

Recognition

Related to Management

Override of Controls

Related to Nature of

Planned Procedures

Related to Extent of

Planned Procedures

Related to Timing of

Planned Procedures

“Revenue recognition -

cutoff, deferred revenue &

deferred costs” (Manager,

Treatment group)

“Revenue Recognition - in

relation to cost estimation”

(Manager, Control group)

“Misappropriation of

America Recovery

Reinvestment Act funds”

(Senior, Control group)

“Revenue - complex

accounting, including

multiple-element sales and

percentage of completion

valuation and timing of rev.

recognition (cut-off)”

(Senior, Control group)

“Changes in specific

customer contracts may cause

errors in financial reporting

and/or give management of

significant entities an

opportunity to manipulate

revenue recognition policy to

meet financial forecasts”

(Senior, Control group)

“Dominance at executive

level by one particular

person could lead to

manipulation in order to

meet set targets.” (Manager,

Treatment group)

“Unsubstantiated JE's

related to revenue and/or

expense” (Manager, Control

group)

“Small operations,

Controller sees a lot,

management override of

controls” (Senior, Treatment

group)

“Management override of

controls via post-close or

top side journal entries”

(Senior, Treatment group)

“Manipulation of earnings

via inappropriate manual,

top-side entries” (Senior,

Control group)

Management override of

controls through manual

journal entries (multiple

mentions of several

variations)

“Detailed contract review on

significant [revenue]

contracts” (Manager,

Treatment group)

“Analysis of [inventory mark-

down reserve account]

balance throughout the year

and prior year” (Manager,

Treatment group)

“Discuss w/ management

level of bulk purchases and

inventory monitoring”

(Manager, Treatment group)

“A/R confirmations”

(Manager, Control group)

“[Test the following control:]

contracts not activated until

dually signed and no material

written changes” (Senior,

Treatment group)

“Review for fraud incentives

arising from covenant regs,

reg environment, earnings

estimates, etc.” (Senior,

Treatment group)

“Revenue cut-off testwork -

detail testing” (Senior,

Control group)

“Scoped in sales and A/R

for a new location”

(Manager, Treatment

group)

“Review of smaller-mid

size bank accounts for

evidence of improper

review/cut-offs” (Manager,

Control group)

“[firm] is testing all sales

transactions above $200k.

Additionally a sample is

tested for the remaining

population.” (Senior,

Treatment group)

“Increase extent of testing

of derivative financial

instruments & contract

review (i.e., classification

of contract as derivative or

accrual acct.)” (Senior,

Control group)

“Vouch 100% of revenue

received to cash payments”

(Senior, Control group)

“In AP, will select items

specifically from new

facility to test cutoff”

(Senior, Control group)

“Test [long-term agreements]

quarterly” (Manager,

Treatment group)

“Test closer to or at YE for

certain procedures”

(Manager, control group)

“Review significant

agreements quarterly.”

(Senior, Treatment group)

“Changing procedure timing

or scoped components from

prior year” (Senior, Control

group) (also relates to extent)

“Perform receivable

confirmations at an interim

date.” (Senior, Control

group)

“We will also perform a

detailed cut-off test at interim

(5 before and 5 after).

Typically, we would [only]

test 5 before and 5 after year

end for cut-off.” (Senior,

Control group)

49

TABLE 4

Descriptive Statistics

Dependent Variables and Control Variables

Panel A: Dependent Variables

Overall

[n=148]

Treatment

[n=71]

Control

[n=77]

Process Variablesa n Mean

Std.

Dev. Med Min Max

n Mean

Std.

Dev. Med

n Mean

Std.

Dev. Med

Statb

p-valueb

PS_CHG_GENERAL 146 0.58 0.87 0.00 -2.00 3.00

70 0.73 0.91 0.00

76 0.45 0.82 0.00

1.99

0.02

PS_CHG_SPECIFIC 148 0.68 1.00 0.00 -2.00 4.00

71 0.87 1.11 1.00

77 0.51 0.87 0.00

2.16

0.02

DISCUSSION_MGT 148 7.12 1.64 7.00 2.00 10.00

71 7.17 1.71 7.00

77 7.07 1.58 7.00

0.36

0.36

DISCUSSION_RESP 148 7.03 2.02 7.00 1.00 10.00

71 6.73 2.15 7.00

77 7.30 1.86 8.00

-1.08

0.09

SESSION_LENGTH 148 34.83 29.75 30.00 5.00 200.00

71 39.44 36.01 30.00

77 30.58 21.90 25.00

1.79

0.04

Fraud Risk Factor Outcome Variablesa

RISKS_NUMBER 145 3.43 1.98 3.00 1.00 11.00

69 3.61 1.82 3.00

76 3.28 2.11 3.00

1.02

0.16

RISKS_NEW 145 0.52 0.94 0.00 0.00 4.00

69 0.72 1.11 0.00

76 0.33 0.70 0.00

2.54

0.01

%REV_REC 147 0.30 0.29 0.33 0.00 1.00 70 0.32 0.34 0.27 77 0.29 0.23 0.33 0.79 0.43

%OVERRIDE 147 0.19 0.22 0.14 0.00 1.00 70 0.18 0.22 0.14 77 0.20 0.22 0.09 -0.62 0.54

Fraud Risk Response Outcome Variablesa

PROC_NUMBER 139 5.55 4.07 4.00 1.00 18.00

66 5.50 3.94 4.00

73 5.59 4.21 4.00

-0.13

0.45

PROC_NEW 139 0.42 1.01 0.00 0.00 7.00

66 0.58 1.25 0.00

73 0.29 0.70 0.00

1.65

0.05

PROC_UNPRED 139 1.72 2.71 1.00 0.00 25.00

66 1.47 3.29 1.00

73 1.95 2.05 2.00

-1.01

0.16

TAILOR 148 0.20 0.40 0.00 0.00 1.00 71 0.14 0.35 0.00 77 0.25 0.43 0.00 2.00 0.08

%RELATE_NATURE 404 0.86 0.32 1.00 0.00 1.00 207 0.88 0.30 1.00 197 0.85 0.34 1.00 1.15 0.25

%RELATE_EXTENT 404 0.13 0.31 0.00 0.00 1.00 207 0.11 0.28 0.00 197 0.14 0.33 0.00 -1.15 0.25

%RELATE_TIMING 404 0.02 0.11 0.00 0.00 1.00 207 0.00 0.03 0.00 197 0.03 0.16 0.00 -2.37 0.02 a See variable definitions in Appendix A.

b Continuous variable statistics are t-statistics from means tests. Indicator variable statistics are Chi-Squared statistics with Yates' continuity corrections. All p-values are two-tailed, except

those with directional predictions.

50

TABLE 4

Descriptive Statistics

Dependent Variables and Control Variables

Panel B: Control Variables

Overall

[n=148]

Treatment

[n=71]

Control

[n=77]

Control Variablesa

n Mean

Std.

Dev. Med Min Max

n Mean

Std.

Dev. Med

n Mean

Std.

Dev. Med

Statb

p-valueb

CLIENT_SIZE 146 3.02 1.30 3.00 1.00 5.00

69 2.80 1.33 3.00

77 3.22 1.25 4.00

-1.97

0.05

PUBLIC 148 0.62 0.49 1.00 0.00 1.00

71 0.52 0.50 1.00

77 0.71 0.45 1.00

5.07

0.02

INDUSTRY_FS 148 0.29 0.46 0.00 0.00 1.00

71 0.21 0.41 0.00

77 0.36 0.48 0.00

3.45

0.06

INDUSTRY_GV/NP 148 0.07 0.25 0.00 0.00 1.00

71 0.00 0.00 0.00

77 0.13 0.34 0.00

7.94

0.00

INDUSTRY_MFG 148 0.25 0.43 0.00 0.00 1.00

71 0.30 0.46 0.00

77 0.21 0.41 0.00

1.09

0.30

INDUSTRY_MISC 148 0.39 0.49 0.00 0.00 1.00 71 0.47 0.50 0.00 77 0.31 0.47 0.00 -1.92 0.06

INHERENT_RISK 148 4.71 2.00 5.00 1.00 10.00

71 4.55 2.07 4.00

77 4.86 1.94 5.00

-0.94

0.35

FRAUD_RISK 148 4.13 1.74 4.00 1.00 9.00

71 4.17 1.93 4.00

77 4.09 1.56 4.00

0.27

0.79

COMPLEXITY 146 0.85 0.38 0.75 0.33 2.00

69 0.86 0.41 1.00

77 0.84 0.34 0.75

0.37

0.71

TEAM_EXPERTISE 148 7.75 1.61 8.00 0.00 10.00

71 7.67 1.81 8.00

77 7.83 1.42 8.00

-0.60

0.55

PTR_CLIENT_EXPC 148 2.63 0.61 3.00 1.00 3.00

71 2.61 0.64 3.00

77 2.65 0.58 3.00

-0.43

0.67

MGR_CLIENT_EXPC 148 36.95 28.04 32.00 0.00 109.00 71 33.23 31.68 24.00 77 40.38 23.90 38.00 1.56 0.12

SR_CLIENT_EXPC 148 23.12 13.94 23.00 0.00 51.00 71 20.61 14.19 18.00 77 25.44 13.39 24.00 2.13 0.04

EXPERIENCE_FFR 147 1.24 0.47 1.00 1.00 3.00

71 1.24 0.49 1.00

76 1.24 0.46 1.00

0.03

0.97

MANAGER1 148 0.51 0.50 1.00 0.00 1.00 71 0.49 0.50 0.00 77 0.52 0.50 1.00 0.32 0.75

FORMAL_FORMAT 148 0.22 0.42 0.00 0.00 1.00

71 0.20 0.40 0.00

77 0.25 0.43 0.00

0.28

0.60 a See variable definitions in Appendix A.

b Continuous variable statistics are t-statistics from means tests. Indicator variable statistics are Chi-Squared statistics with Yates' continuity corrections. All p-values are two-tailed, except

those with directional predictions.

51

TABLE 5

Regression Results: Process Variables

Dependent Variables: PS_CHG_GENERAL, PS_CHG_SPECIFIC, DISCUSSION_MGT, DISCUSSION_RESP,

SESSION_LENGTH b

(1)

PS_CHG

_GENERAL

(2)

PS_CHG

_SPECIFIC

(3)

DISCUSSION

_MGT

(4)

DISCUSSION

_RESP

(5)

SESSION_

LENGTH

Variablea Pred. Coefficient Coefficient Coefficient Coefficient Coefficient

Sign (t-statistic)c (t-statistic)

c (t-statistic)

c (t-statistic)

c (t-statistic)

c

TREATMENT + 0.33** 0.31** 0.36* -0.43 10.46**

(1.95) (1.74) (1.31) (-1.23) (1.84)

Control Variables

CLIENT_SIZE

+ 0.05 0.14** 0.29*** 0.03 7.11**

(0.68) (1.74) (2.41) (0.17) (1.97)

PUBLIC + -0.33* -0.52** 0.50** -0.23 -0.08

(-1.77) (-2.52) (1.68) (-0.60) (-0.02)

INDUSTRY_FS +/- 0.29 0.25 0.07 0.14 5.42

(1.44) (1.14) (0.17) (0.28) (0.85)

INDUSTRY_GV/NP +/- 0.47 0.20 1.01* 1.42** 2.59

(1.05) (0.47) (1.90) (2.39) (0.29)

INDUSTRY_MFG +/- 0.12 0.12 0.44* 0.21 18.32**

(0.63) (0.68) (1.29) (0.52) (2.05)

INHERENT_RISK + -0.02 -0.07 -0.22*** -0.10 1.35

(-0.45) (-1.28) (-2.74) (-0.85) (0.81)

FRAUD_RISK + 0.03 0.07* 0.17** 0.36*** 1.02

(0.52) (1.31) (1.93) (2.95) (0.45)

COMPLEXITY + 0.05 0.03 0.19 -1.52** 6.26

(0.21) (0.17) (0.43) (-2.52) (0.69)

TEAM_EXPERTISE +/- 0.02 0.09 0.17** 0.18** 1.53

(0.42) (1.63) (2.31) (2.10) (0.83)

PTR_CLIENT_EXPC +/- -0.21* -0.13 -0.34* -0.06 3.08

(-1.77) (-0.93) (-1.96) (-0.22) (0.74)

MGR_CLIENT_EXPC +/- 0.00 0.00 0.01* 0.01* -0.12

(0.35) (-0.64) (1.84) (1.93) (-1.13)

SR_CLIENT_EXPC +/- 0.00 -0.01* -0.03** -0.01 -0.29

(-0.47) (-1.67) (-2.53) (-1.07) (-0.88)

EXPERIENCE_FFR + -0.14 -0.14 0.26 0.49* 4.49

(-1.08) (-0.91) (1.06) (1.64) (0.97)

MANAGER1 +/- -0.50*** -0.41** 0.75*** 0.41 -0.630

(-3.94) (-2.40) (2.89) (1.25) (-0.18)

FORMAL_FORMAT +/- 0.17 -0.12 0.15 0.58 16.21**

(1.06) (-0.68) (0.43) (1.62) (2.21)

CONSTANT +/- 1.13 0.66 4.93*** 5.05*** -31.24

(1.57) (0.90) (5.10) (3.41) (-0.89)

Observations 143 145 145 145 146

Adjusted R2 .099*** .118*** .179*** .159*** 0.186

a See variable definitions in Appendix A.

b All dependent variables are continuous and are analyzed using OLS.

c Numbers in parentheses are t-statistics based on Rogers standard errors, which are clustered at the engagement level and

control for serial correlation and heteroskedasticity (Petersen 2009).

*, **, and *** represent significance at 10%, 5%, and 1%, respectively. All p-values are two-tailed except those with

directional predictions.

52

TABLE 6

Regression Results: Fraud Risk Factor Outcome Variables

Dependent Variables: RISKS_NUMBER, RISKS_NEW, %REV_REC, %OVERRIDEb

(6)

RISKS_

NUMBER

(7)

RISKS_

NEW

(8)

%REV

_REC

(9)

%OVER

RIDE

Variablea

Pred. Coefficient Coefficient Coefficient Coefficient

Signd (t-statistic)

c (t-statistic)

c (t-statistic)

c (t-statistic)

c

TREATMENT +

d 0.59** 0.38** -0.01 -0.040

(1.75) (2.10) (-0.24) (-0.82)

Control Variables

CLIENT_SIZE

+/- -0.07 -0.20** -0.02 -0.01

(-0.28) (-2.15) (-0.91) (-0.35)

PUBLIC +/- 0.10 0.07 0.07 0.02

(0.24) (0.39) (0.98) (0.52)

INDUSTRY_FS +/- 0.32 0.22 -0.25*** -0.10*

(0.53) (0.80) (-4.17) (-1.95)

INDUSTRY_GV/NP +/- 0.45 0.44 -0.07 -0.01

(0.79) (1.51) (-0.77) (-0.13)

INDUSTRY_MFG +/- 0.64 0.68*** 0.01 0.00

(1.48) (3.19) (0.18) (0.06)

INHERENT_RISK +/- 0.33** 0.13** -0.02 0.00

(2.33) (2.21) (-1.38) (-0.33)

FRAUD_RISK +/- -0.19 0.00 0.01 0.01

(-1.13) (-0.02) (0.37) (0.99)

COMPLEXITY +/- 0.32 -0.42 -0.12* -0.02

(0.57) (-1.52) (-1.72) (-0.30)

TEAM_EXPERTISE +/- -0.10 0.02 -0.02 0.01

(-0.69) (0.37) (-0.91) (1.28)

PTR_CLIENT_EXPC +/- 0.09 -0.30** -0.02 0.10***

(0.29) (-2.05) (-0.66) (2.93)

MGR_CLIENT_EXPC +/- 0.01 0.00 0.00 0.00

(1.34) (0.29) (-0.44) (-0.29)

SR_CLIENT_EXPC +/- 0.00 -0.01 0.00 0.00

(-0.07) (-1.33) (0.01) (1.32)

EXPERIENCE_FFR +/- -0.31 -0.07 -0.04 -0.01

(-0.80) (-0.44) (-0.93) (-0.40)

MANAGER1 +/- -0.16 -0.10 0.07* 0.05*

(-0.52) (-0.78) (1.81) (1.72)

FORMAL_FORMAT +/- 0.37 -0.09 0.04 0.06

(0.85) (-0.58) (0.86) (1.43)

CONSTANT +/- 2.59 1.28* 0.84*** -0.21

(1.52) (1.73) (3.44) (-1.29)

Observations 142 142 144 144

Adjusted R2 .019** .189** .216*** .068**

a See variable definitions in Appendix A.

b All dependent variables are continuous and are analyzed using OLS.

c Numbers in parentheses are t-statistics based on Rogers standard errors, which are clustered at the

engagement level and control for serial correlation and heteroskedasticity (Petersen 2009). d We predict a positive coefficient on the TREATMENT variable in the models with RISKS_NUMBER

and RISKS_NEW as dependent variables. We do not make directional predictions for the

TREATMENT variable in the models with %REV_REC and %OVERRIDE as dependent variables.

*, **, and *** represent significance at 10%, 5%, and 1%, respectively. All p-values are two-tailed

except those with directional predictions.

53

TABLE 7 Regression Results: Fraud Risk Response Outcome Variables

Dependent Variables: TAILOR, PROC_NUMBER, PROC_NEW, PROC_UNPRED, %RELATE_NATURE, %RELATE_EXTENT,

%RELATE_TIMINGb

(10)

PROC_

NUMBER

(11)

PROC_

NEW

(12)

PROC_

UNPRED

(13)

TAILORa

(14)

%RELATE

_NATURE

(15)

%RELATE

_EXTENT

(16)

%RELATE

_TIMING

Variablea

Pred. Coefficient Coefficient Coefficient Odds Ratiob Coefficient Coefficient Coefficient

Sign (t-statistic)c (t-statistic)

c (t-statistic)

c (z-statistic)

c (t-statistic)

c (t-statistic)

c (t-statistic)

c

TREATMENT +/- -0.13 0.13 -0.24 0.11*** 0.09* -0.09** -0.03**

(-0.16) (0.57) (-0.40) (-3.15) (1.86) (-2.03) (-2.04)

Control Variables

CLIENT_SIZE +/- -0.10 -0.18 0.06 1.08 0.00 0.01 0.01

(-0.23) (-1.51) (0.35) -0.19 (-0.11) (0.46) (1.02)

PUBLIC +/- 0.59 -0.25 0.69 0.79 0.08 -0.08* 0.00

(0.65) (-1.17) (1.63) (-0.31) (1.61) (-1.70) (-0.13)

INDUSTRY_FS +/- -0.76 0.07 -0.06 0.06** -0.03 0.02 0.00

(-0.69) (0.36) (-0.09) (-2.30) (-0.45) (0.44) (-0.08)

INDUSTRY_GV/NP +/- 0.31 -0.01 0.23 NA

d 0.15** -0.16*** -0.04

(0.18) (-0.04) (0.30) NAd (2.41) (-2.64) (-1.48)

INDUSTRY_MFG +/- 0.98 0.61 0.27 1.51 -0.04 0.05 0.00

(0.87) (1.62) (0.39) -0.53 (-0.80) (1.02) (0.33)

INHERENT_RISK +/- 0.37 0.02 0.03 1.13 -0.02 0.02 0.00

(1.47) (0.27) (0.23) -0.57 (-1.37) (1.46) (0.17)

FRAUD_RISK +/- 0.14 0.06 -0.06 0.76 0.02 -0.02 -0.01**

(0.39) (1.12) (-0.33) (-1.38) (1.63) (-1.23) (-2.27)

COMPLEXITY +/- -0.58 -0.41* -1.26** 6.09* -0.06 0.10** 0.00

(-0.52) (-1.69) (-2.40) 1.67 (-1.18) (2.15) (-0.05)

TEAM_EXPERTISE +/- 0.00 0.03 0.30* 1.17 -0.02 0.01 0.00

(-0.01) (0.65) (1.94) -0.64 (-1.24) (1.12) (1.17)

PTR_CLIENT_EXPC +/- -0.48 0.10 0.01 -0.44* 0.04 -0.03 -0.01

(-0.73) (0.66) (0.02) (-1.87) (1.08) (-0.77) (-0.65)

MGR_CLIENT_EXPC +/- 0.02 0.00 0.01 1.02 0.00 0.00 0.00

(1.15) (0.47) (0.81) -1.42 (-1.00) (0.96) (-0.37)

SR_CLIENT_EXPC +/- -0.02 0.00 -0.01 0.99 0.00 0.00 0.00

(-0.84) (-0.61) (-0.67) (-0.32) (0.98) (-1.21) (-1.11)

EXPERIENCE_FFR +/- 0.21 -0.21 0.18 1.13 -0.02 0.02 0.00

(0.28) (-1.37) (0.41) -0.21 (-0.43) (0.58) (0.16)

MANAGER1 +/- -0.20 0.03 -0.24 1.25 0.07* -0.06* 0.00

(-0.31) (0.26) (-0.51) -0.76 (1.71) (-1.67) (0.35)

FORMAL_FORMAT +/- -0.18 -0.11 0.11 0.34* -0.01 0.01 0.01

(-0.19) (-0.62) (0.19) (-1.67) (-0.20) (0.26) (0.72)

CONSTANT +/- 4.86 0.74 -0.35 0.46 0.86*** 0.03 0.05

(1.28) (1.10) (-0.26) (-0.19) (3.80) (0.12) (1.48)

Observations 136 136 136 135 400 400 400 R

2 measure

e .024** 0.075 .005** .300** .030* .033** 0.018

a See variable definitions in Appendix A.

b PROC_NUMBER, PROC_NEW, PROC_UNPRED, %RELATE_NATURE, %RELATE_EXTENT, and %RELATE_TIMING are continuous

dependent variables and are analyzed using OLS. TAILOR is a dichotomous variable (see Appendix A) and model (11) is estimated using

logistic regression; we report Odds Ratios and z-statistics for this model. c Numbers in parentheses are t-statistics or z-statistics, as noted, based on Rogers standard errors, which are clustered at the engagement level

and control for serial correlation and heteroskedasticity (Petersen 2009). d None of the 10 observations from the Government/Not-for-profit industry eliminated fraud risk procedures used in the prior year. This

indicator variable is therefore excluded from model (10) due to lack of variation on the dependent variable. e Adjusted R

2 reported for OLS models; Pseudo R

2 reported for logit model.

*, **, and *** represent significance at 10%, 5%, and 1%, respectively. All p-values are two-tailed.