IRBA Root Cause Analysis Information Session - SAICA Information... · IRBA Root Cause Analysis Information Session SAICA Offices, JHB ... Case Studies 9) ... The root cause that

IRBARoot Cause Analysis Information Session

SAICA Offices, JHB27 June 2017

Presenters:

Imre Nagy, Pieter Cloete

& Sadhir Issirinarain

2

Root Cause Analysis (RCA)

Information Session

27 June 2017

RCA Information Session

3

Index

1) Statistics

2) The need for RCA

3) Policy

4) The IRBA Remediation Process

5) RCA Introduction & Background

6) Principles of RCA

7) Questions

8) Case Studies

9) Contact

Statistics

4

International

Financial Reporting Council (FRC)

“We expect all the firms we inspect to make continuous improvements such

that, by 2019, at least 90% of FTSE 350 audits reviewed will be assessed

as requiring no more than limited improvements.” Effective RCA should

help to achieve this.

International Forum of Independent Audit Regulators (IFIAR)

Although the 2016 report showed only 1% improvement from 2015 on

engagements with at least 1 finding, IFIAR’s Global Audit Quality (GAQ)

Working Group and the GPPC networks undertook an initiative aimed to

reduce the frequency of inspection findings by at least 25%, on an

aggregate basis across the GPPC networks over four years through

RCA.

Statistics (cont …)

5

Local

• The IRBA aims to promote a notable reduction in inspection findings

through its Remedial Action Process (RAP) with the firms.

• The IRBA has actively engaged with 136 (72%) auditors that received

an unsatisfactory inspection result in the 2016 year (2016 Public

Inspections Report).

• The IRBA has actively engaged with 55 auditors that received an

unsatisfactory inspection result in the past six months.

o 23 of the 55 (42%) did not identify the true root cause or possible

causes were too vague.

o 8 of the 55 (15%) did not implement an appropriate action plan to

address root causes.

• However, since the introduction of the remedial action process by the

IRBA 2,5 years ago, approximately 80% of recent re-inspections

received a satisfactory result, which is encouraging.

The Need for RCA

6

• There is a significant misunderstanding of the RCA and in many

instances it would either be incorrectly prepared or not prepared at all.

• Identifying, for example, “lack of documentation” as the root cause

without getting to the real answer as to WHY the documentation was

deficient is not enough.

• As per the 2016 IRBA Public Inspections report, the highest root cause

allocation was “human error”, without drilling down to exactly WHY the

issue existed or resulted in a finding (root cause vs. symptoms).

• Those auditors that effectively identified the underlying root causes and

implemented real proactive action plans demonstrated significant

improvement during follow-up inspections.

7

Policy Reference

Policy

• In its efforts to promote high audit quality, the IRBA adopted the IFIAR

Core Principle 11, which states that audit regulators should have a

mechanism for reporting inspections findings to the audit firm and

ensuring remediation of findings with the audit firm.

• ISQC 1 para 50: “The firm shall communicate to relevant engagement

partners and other appropriate personnel deficiencies noted as a result

of the monitoring process and recommendations for appropriate

remedial action.” (Ref: Para. A69)

• ISA 220 A33: “In considering deficiencies that may affect the audit

engagement, the engagement partner may have regard to measures the

firm took to rectify the situation that the engagement partner considers

are sufficient in the context of that audit.”

8

Policy Reference (cont …)

• Current IAASB agenda (WIP project) on enhancing audit quality: “We

are seeking to understand what our stakeholders think about the ‘root

causes’ of these inspection findings.” (Refer to the IAASB website)

• Proposed revision to ISQC 1: For example, include remediation as a

header as well as refer to internal and external monitoring

findings.

• Determine the cause(s) of deficiencies and evaluate their effect and

implement appropriate remedial action(s).

• International buzz word: Professional Scepticism(same principle can be applied in conjunction with RCA!)

o Professional scepticism means an attitude that

includes a questioning mind; being alert to

conditions that may indicate possible misstatement

due to error or fraud; and a critical assessment of evidence.

The IRBA Remediation Process

9

South African Approach – Remedial Action Process (RAP)

The IRBA Remediation Process

(cont …)

10

Please note:

• The IRBA will visit all unsatisfactory/investigation referral inspection

results for engagements and firms.

• Request further documentation and/or evidence that the condition/s was

met for conditional satisfactory results. (Conditional results can be

motivated where there are only a few findings not impacting materially on

the audit report and not significant risk or public interest in nature.)

• The evidence is reviewed and assessed on whether the condition/s

was met, and these practitioners are also visited where deemed

necessary.

• If the condition/s is not met, the file is referred back to the Inspections

Committee for reconsideration.

• The sole objective is to prompt remedial action by the firm on all of its

audits.

RCA Introduction & Background

11

Sources

Local:

• IRBA Remedial Action Process (RAP) feedback

• 2016 Public Inspections Report (refer to the

IRBA website)

• IRBA Internal Policy and Procedure (not public)

International:

• IFIAR Inspection Survey Report (www.ifiar.org)

• FRC audit quality thematic review report (www.frc.org.uk)

• ICAEW report on “Improving audit quality using root cause analysis”

(www.icaew.com)

12


(cont …)

Brief history on RCA:

• Developed by Sakichi Toyoda who later founded Toyota Motor Company.

• RCA was first used during the development of Toyota’s manufacturing

processes in 1958.

Definitions:

• Root Cause: The original event, action and/or condition generating an

actual or potential undesirable condition, situation, nonconformity or failure.

• RCA is a technique for identifying the underlying key cause (or causes)

behind review findings, whether specific to one audit or firm wide, so

that an appropriate and achievable action (or actions) can be taken to

prevent the recurrence of negative outcomes and promote the recurrence of

positive ones.

13


(cont …)

14


(cont …)

Visible

Not visible


(cont …)

RCA advantages

• Leads to consistent audit quality and reduction of inspection

findings.

• Adds value to the firm, partners and staff.

• There is the potential for cost reduction.

• Provides a learning process for better understanding of relationships

(cause and effect; and solutions).

• Provides a logical approach to problem solving using data that

already exists.

• Reduces risk.

• Prevention of recurring failures.

• Improved performance.

• Leads to more robust systems, policies and procedures.

15

16


(cont …)

Overall objective of RCA

Continuous improvement of audit quality within the firm

Audit

Quality

Improvement

Process

Principles of RCA17

The RCA cycle

Principles of RCA (cont …)

The following are examples of possible causes, but they do NOT

drill down to the true ROOT cause:- Human error

- Misunderstood the requirement

- Misunderstood the standards

- Documentation

- System error/Audit methodology

- Elaborating on the finding

- Expanding on symptoms

- The consultant messed up

- It was an oversight

- Audit procedures/software not updated, etc.

- Should find and address the real Root Cause.

- Should, therefore, address the question:

“WHAT in the system failed to make the

problem occur?”

- Until the root cause is identified, keep

asking: WHY?

18


Why do we have poor root cause analyses?

- We have not set criteria about what makes an acceptable corrective action

plan (firms to implement RCA and RAP Policies and Procedures).

- We continue to accept bad answers.

- People (internal and external) do not have the Root Cause Analysis culture;

don’t know any process or are not effectively trained.

- We are addressing the symptoms and not the true root of the issue.

Steps that can address poor RCA (DMAGIC):

Define the problem.

Map the process and collect data.

Analyse data and identify causes.

Gather information and evidence (asking why) until the true root cause is clear.

Implement an action plan to address the root cause(s).

Control and monitor the plan.

19


20

Using the 5 Cs

1) Criteria:

The law, regulation, contractual obligation, policy, procedure or

best practice that is expected to be followed.

2) Condition:

The factual analysis of the process as it exists.

3) Consequence/Effect:

Why the issue is important and noteworthy from a compliance,

financial, or operational standpoint.

4) Cause:

The root cause that allowed the condition to not emulate the

criteria.

5) Corrective Action/Recommendation:

Change that will address the root cause – action plan.

Principles of RCA (cont …) 21

It is important to get your mind into an investigative mode (refer to principle of professional scepticism).

Investigating checklist example:

• Identify the procedure owner and those involved.• Gather information:

- Data- Employee Input- Flowcharts of the process- Procedures- Records (quantitative data)

• Has the problem occurred in the past?• Identify the root cause.

Rules of the investigation:

• Use proven root cause analysis tools.

• Think “out of the box”.

• Brainstorm.

• Take the time needed.

• Put a plan together.


RCA techniques:

• The 5 WHYs (today’s focus).

• Failure mode and effects analysis.

• Flowcharting of the process flow, system flow and data flow.

• Fishbone diagrams.

22


• The objective is to learn, train and improve.

• NB! Take responsibility, accountability and action.

23


The 5 WHYs explained …

24


25

Real life examples…


26


Example of 5 WHYs in action (audit)

Finding: Insufficient audit work performed in relation to a significant audit risk

First why The audit staff member performing the work

did not adequately understand what needed

to be done to address the audit risks.

Second why The audit manager was not briefed by the

audit partner and so could not adequately brief

those performing the work.

Third why The audit partner did not adequately brief the

staff member on the audit strategy.

Fourth why The audit partner was not around at the

planning stage of the audit.

Fifth why The audit partner had other commitments

and therefore wasn’t able to be around to

brief the audit team.

27


Example of 5 WHYs in action (audit)

Finding: Failure to identify cash flow statement misclassification

First why The audit staff member performing the work did

not practice due care or there was a lack of

understanding of the subject matter.

Second why There was a lack of

direction/supervision/motivation or

late/inadequate documentation because of time

pressure.

Third why Fee pressure/resource shortage or poor project

management.

Fourth why Inexperienced senior manager.

Fifth why The audit partner did not provide

training/guidance and did not apply his time

to supervise the manager.

28


Keep environmental matters in mind (most root causes can be

traced back to decisions, actions or inactions by one or more

engagement team members):

• Leadership and culture

• Competency of personnel

• Hiring qualified personnel

• Lack of or insufficient training

• Adequacy of technology or tools

• Departmental morale

• Resources (budget/personnel)

• Decision-making authority

29


Root cause examples

30

Resource issues:

• Competency of staff (can impact personal, ethical and attitude issues).

• Experience of staff.

• Engagement team dynamics (e.g. incompatible combinations, skills or

experience gaps, lack of continuity or over-familiarity).

• Time available (rushed jobs are rarely successful).

• Number of staff available (understaffed jobs are rarely successful).

• Lack of clarity on competencies and responsibilities.

31

Personal, Ethical and Attitude issues

• A mindset that is prepared to cut corners (perhaps due to complacency

or a desire to keep to original timetables and budgets).

• Unwillingness to acknowledge or learn from mistakes.

• Being unwilling or unable to direct, supervise or review effectively,

even when resources are available and procedures require this.


Principles of RCA (cont …)32

Process issues

• Issues arising from the firm’s policies and procedures – are they well

framed; are there gaps; are they well understood and complied with?

• Does on-the-job mentoring and reviewing happen in the way they

should?

• Does the staff appraisal system drive improvement or is it cosmetic?

• Failure to consult when appropriate.

• Poor project management, incl. leaving issues to the end of the audit.

People

Policy


Leadership issues

• Do staff receive appropriate leadership?

• Is change effected when required?

• Is leadership cosmetic or real? (Do actions belie words?)

Client issues

• Can the firm be fairly expected to serve its client base? (Consider:

competence, resources and specialities.)

• Are review findings rooted in difficulties with client interaction?

For example, fee pressure, an unreasonable client-imposed

deadline to complete the audit, poor quality or information arriving

late from the client.


Link to ISQC 1 Para. 26: “The firm shall establish policies and

procedures for the acceptance and continuance of client

relationships and specific engagements, designed to provide the

firm with reasonable assurance that it will only undertake or

continue relationships and engagements where the firm:

(a) Is competent to perform the engagement and has the

capabilities, including time and resources, to do so; (Ref: Para.

A18, A23)

(b) Can comply with relevant ethical requirements; and

(c) Has considered the integrity of the client, and does not have

information that would lead it to conclude that the client lacks

integrity. (Ref: Para. A19-A20, A23)”

•


Remedial Action Plan (RAP) examples

35

When root causes are identified correctly, it is easy

to develop and implement an action

plan as follows:

• Reviewing of resourcing.

• Improving the project management.

• Increased focus on joiners or leavers.

• Coaching and guidance on related initiatives.

• Improving the integration of internal experts.

• Real-time monitoring/support teams.

• Methodology enhancements.

• Guidance and communications.

• Training (technical and soft skills).

• Supervision and review.

• Software, procedures and technical updates.

36

Questions?

Case Studies

Presented by

Imre Nagy – Director Inspections

Please refer to separate slides

37

38

Contact us:

Pieter Cloete

SM Remedial Action

IRBA

[email protected]

+27 87 940 8794

mailto:[email protected]

Documents

IRBA Root Cause Analysis Information Session - SAICA Information... · IRBA Root Cause Analysis Information Session SAICA Offices, JHB ... Case Studies 9) ... The root cause that