Integrated Auditing of (IT) Application - Bitterli … · Integrated Auditing of IT Applications ... designed to provide reasonable assurance ... Financial Management module may not

Integrated Auditing

of (IT) ApplicationEuroCACS 2009 Session 211

Peter R. Bitterli, CISA, CISMhttp://www.bitterli-consulting.ch

[email protected]

Please observe the copyright: You are allowed to use and further

distribute this presentation only with this copyright notice attached. If you

use parts of this documentation in presentations or other diagrams you

have to refer to the source. Any commercial use of this presentation is

only allowed with written consent of the author.

Abstract (I)Integrated Auditing of IT Applications

Of course, all auditors know exactly how to audit any (IT)

application. At least, that is what they keep telling their

superiors and also their clients. Reality, however, shows a

different picture: Although all major audit companies and also

some audit organization (e.g. ISACA) have a specific

approach to auditing (IT) applications, many auditors out

there in the tough world of real auditing have no idea on how

to start. And if they do start, there is often no coordination

and collaboration between the financial auditor and the IT

auditor, making auditing a highly risking task.

Abstract (II)Integrated Auditing of IT Applications

This session shows a step-by-step approach that has been

developed by the IT Standards Board of the Swiss Institute of

Certified Accountants and Tax Consultants. The “new”

approach has been published in German and also in French

but is in general valid for all types of application auditing in

possibly most countries in the world. The audit approach is

consistent with recently published standards and related

information, such as “IT Controls for Sarbanes Oxley”, but

aims to be easily understandable also by less experienced

auditors.

Learning ObjectivesThe participants will learn about …

The participants will learn about …

• why some of the typical approaches to auditing are a

waste of resources.

• how an integrated approach to application audit

works that combines both financial and IT auditing.

• which audit steps are necessary in such a combined

approach.

• why this audit approach usually leads to a lower

overall audit risk.

Introduction

Increasing ChallengesAuditors have a tough job

• Auditing a company’s financial statements

presents financial auditors with an increasing

number of challenges:

– Rapid development in accounting standards

– Increasingly automated “production” of financial

data

– More and more complex information systems

Value-for-money AuditingAuditors have a tough job

• Audit is required to deliver value-for-money

• Apart from the highly proficient performance

expected from audit staff and management

anyhow, risk-orientation during all audit steps

helps:

– to stay focussed on the critical items and thereby

not waste audit resources (i.e. efficiency)

– to (indirectly) ensure adequate controls for high-

risk situations (i.e. effectiveness)

Risk-oriented AuditingAuditors have a tough job

• It is broadly recognized (and sometimes

required by regulatory bodies) that audit

should be risk-oriented, e.g.

– during the annual audit planning

– during the strategic planning phase for an

individual audit

– during each and every activity of an individual

audit

Risk-oriented

Auditing

• Swiss “Handbook for

the auditing of financial

statements” (HWP):

– Currently valid approach

to auditing (1996)

– 4 major audit phases

– Risk assessment

• is part of audit planning

• is mostly performed by

financial auditors

Risk-oriented AuditingAudit risk is never equal zero

Substantive

Testing

Assess-

ment of

Controls

Assess-

ment of

Entity Level

Controls

•Manual Controls

• IT Application

Controls

• ITGC

Source: Andreas Toggwyler, KPMG;

Kammertagung 2007, Bern

Financial Auditors and ITWhere do financial auditors adequately include IT issues in every aspect of auditing?

• Typically, financial auditors

– don’t include IT issues …

• when doing the overall risk analysis

• when planning audits of specific applications

• when performing audits of specific applications

– or – if the do include IT issues – are

• not experienced enough to really understand the

impact of IT on (automated) business applications

• not willing to relinquish a fair share of their budget to

IT audit

Model behind our Audit ApproachA “Creme Slice” with four layers

IT Infrastructure

Applications

Business processes

Basic IT Systems

“Guide to the Audit of IT Applications”Not a standard, instruction or recommendation – just a “professional communication”

• Developed by the“IT Standards Board of theSwiss Institute of CertifiedAccountants and TaxConsultants”

• Written in– German (May ’08)

• Translated into– French (November ’08)

– English (translated byEuropean Court of Auditors;available June ’09)

AuthorsAll authors have been or still are a member of the IT Standards Board

In alphabetic order

• Peter R. Bitterli, Bitterli Consulting

• Jürg Brun, Ernst & Young

• Thomas Bucher, Deloitte & Touche

• Brigitte Christ, Zurich Financial Services, IA

• Bernhard Hamberger, Ernst & Young

• Michel Huissoud, Swiss Federal Audit Office

• Daniel Küng, PWC

• Andreas Toggwyler, KPMG

• Daniel Wyniger, Swisscom IT Services, IA

Coverage of our ApproachThe presented approach to auditing (IT) applications covers the upper two levels

Controls are the policies, procedures,

practices and organizational structures

designed to provide reasonable assurance

that the business objectives will be

achieved and undesired events will be

prevented or detected and corrected.

Definition

Controls

Definition

Internal Control System (ICS)

Entirety of all methods and procedures used

in a company for ensuring the:

– protection of business assets;

– correctness and reliability of bookkeeping;

– efficient management of the company;

– compliance with the business strategy;

– prevention and detection of errors and irregularities.

Objectives of Audit Activity(Source: COBIT)

• To provide reasonable assurance to the management

that the relevant control objectives are achieved

• To identify significant weaknesses in these

controls

• To document the risk connected with such a weakness

• To advise management on the measures to be

implemented

Definition

Controls/Substantive Testing

Controls testing (compliance testing) are

those audit tests which determine if internal

controls are being applied in a manner

described in the documentation and in

accordance with management‘s intents.

Substantive testing are tests of detailedactivities and transactions, or analyticalreview tests, designed to obtain auditevidence on the completeness, accuracy orexistence of those activities or transactionsduring the audit period.

Approach to

Integrated

Auditing

Objectives of our Approach

• For auditing (IT-based) applications:

– Integrated approach of both auditors and

IT auditors

– Adequately taking into consideration all important

aspects – both non-IT as well as IT

– Audit important IT aspects with significant

influence on the audit objectives

– Minimize the audit risk

Our Approach:

8 Recommended Steps

• Analyze financial state-

ments and related risks

– allows to identify the

important applications

• Examine quality of

control system

– design effectiveness

– operational effectiveness

• Audit of general IT

controls not covered!

Step 1:

Analysis of the

balance sheet

Step 1:

Balance Sheet, Profit & Loss

• Objectives

– Establish which items in

the balance sheet and

the ”profit & loss”

account are relevant

– Identify significant trans-

actions (and transaction

classes) that generate

these items

Step 1:

Identify the Relevant Accounts

• Identification of relevant

accounts

– Carry out risk evaluation

– Consider materiality

– Identify risks that may influence the

annual accounts

Major Accounts



Step 1:

Identify Important Transactions

• Identification of transactions

– Analyse which transactions or

transaction classes have had the

greatest influence, e.g.

• Carry out data analysis for

“interesting” entries to specific

accounts to indirectly identify which

transactions made these entries

• Work back from main accounts via

main transactions to underlying

business operations

Major Accounts

Material Transactions

Business Activities



• This backward approach will

exclude the analysis of

unimportant transactions (i.e.

supports audit efficiency)

Step 1:

Reasons for “Backward Approach”

Major Accounts

Material Transactions

Business Activities

Step 2:

Identification of

business processes

and data flow

Step 2:

Business Processes and Data Flows

• Objectives:

– Identify the relevant

business process which

are at the origin of the

transactions and

transaction classes

identified in step 1

– I.e., identify processes

that have direct

influence on the

financial flows

Remark:

A process is defined as a series of

manual, semi-manual or automated

tasks that service to create or process

information, products or services.

Step 2:

Identify Relevant Business Processes

• Identify processes that influence identified

transactions

• Acquire sufficiently detailed understanding of

selected processes

– Routine business processes

– Non-routine financial processes

Step 2:

Questions for Understanding Controls

Some clever questions:

• Who performs the control?

• What exactly is done – and what could go

wrong?

• When, how often and where is the control

performed?

• How will exceptions be treated?

• How will auditability/traceability be

ensured?

Step 2:

Audit Documentation (I)

• Base work on existing documentation, but

amend this where necessary

• Where no documentation is available, the

auditor should build the required

documentation

Step 2:

Audit Documentation (III)

• Document in such a way that the following

types of information are available or can be

amended during later steps:

– Risks

– Key controls

• Also document the processes for handling:

– Parameters

– Master data

Transaction data

Standing Data

Personal data

Payment instructions

HR

Application

Input Data:

# Hours

Expenses

Output Data

Bookings

Salary payments

Other Payments

Application Parameters

Hourly wages

Other Rates

2 Repetitive

Errors

3 Singular

Error

1 Systematic

Errors

Step 2:

Parameters and Standing Data

Source: Michel Huissoud,

Swiss Federal Audit Office;


Step 2:

Audit Documentation (II)

• Choose a suitable level of detail

• Subdivide complex processes into sub-

processes to reduce complexity

• Remember and accept as fact: There are

many different possibilities for documenting

processes, risks and key controls

Step 2:

Documentation: Tables

Example:

Presentation in table form – suited for simple facts and contexts

Step 2:

Documentation: Processes, Interfaces

Example:

Graphical presentation (process model chart with processes and

interfaces) – suited for complex interactions

Step 2:

Documentation: Processes

Example:

Graphical presentation (process model chart with management,

business and support processes) – suited for complex interactions

Step 2:

Documentation: Processes, Interfaces

Example:

Graphical

presentation (flow

chart) – suited for

complex

interactions

Step 3:

Identification of core

applications and the

most important IT-

related interfaces

Step 3:

Core Applications and IT Interfaces

• Objectives:

– Identification of relevant

IT applications

– Identification of

interfaces of the above

applications

– Establish a detailed

audit scope

– Draw up an audit plan

Step 3:

Core Applications and IT Interfaces

• Draw a map

of the core

applications

Step 3:

Inventory of Financial Applications

• Set up an inventory of financial applications

– Standard applications

– Highly adapted standard applications

– Internally developed applications

• Above distinction is necessary as these

categories have very different risk profiles

Step 3:

Standard Applications

• Standard applications:

– Multitude of relevant inbuilt controls, typically for:

• Validation of input

• Management of processing

• Processing of transactions

• Management of expenditures

– Ensure that the expected checks exist and are

applied

Step 3:

Questions for Standard Applications

• For standard applications, ask the questions:

– What types of standard applications does the

company use?

– Are these standard applications usual in the

sector of activity concerned?

– Are the standard applications certified and by

whom?

– Are these standard applications well-established

in the market?

Step 3:

Questions for Standard Applications

• For standard applications, ask questions

(cont.):

– Do information sources on this application exist,

and, if so, are there known security or other

weaknesses?

– Does the auditor have sufficient personal

knowledge and experience of the application?

Access Control




Inventory ClosingMicrosoft® Business Solutions–Axapta® 3.0 and Service Packs

Version: 1.0, published: December, 2004

…”Under certain conditions, Microsoft Axapta 3.0 can return incorrect

inventory values in the general ledger as reported from different modules

within Microsoft Axapta.

Customers may also experience issues with running inventory closing

routines at all, due to one or more code issues that can manifest under

specific conditions.

Finally, the value of the inventory as reported from the Microsoft Axapta

Financial Management module may not match the expected value when

customers perform a physical inventory count.”….




Correctness

CorrectnessSource: Michel Huissoud,









Step 3:

Adapted Standard Applications

• Highly adapted standard applications:

– Although such “components” could be reliable,

more information is required on …

• interaction of these components as such applications

are individually configured for each client

• additional client-specific programming

Technical Environment

Step 3:

Errors in Standard Applications

Core of Standard

Application

Parameters

Errors in the design or programming of

the core functionality

“Add-

ons”

Missing „must“ Functionality

Incompatibility of core to environment

Incompatibility of parameterization of core

Source: Michel Huissoud, Swiss Federal Audit Office;


Errors in user handling?

How resistant to errors

should a standard

application be?

Missing “nice to have“

functionality

Material errors during

parameterization

Errors in the add-ons

Incompatibility of core

and add-ons

?

Step 3:

Audit of “Standard” Applications

• Standard application packages not always

reliable – more so for highly adapted

standard applications

• Therefore get informed about:

– Maturity of software version

– Level of parameterization

– Former audit results (of colleagues?)

– Internet forum

– Release documentation

Step 3:

Internally Developed Applications

• Internally developed applications:

– Generally require (much) more audit work

– To save time, the auditor should base his/her

work (if possible) on documented tests within the

company

• If these tests do not exist or are not usable (i.e. test

design or test documentation lacking, errors not

corrected after tests, no formal acceptance by

users,…), the auditor must perform the necessary

tests himself

Step 3:

Outsourced Business Activities

• Outsourcing of activities or business

processes …

– involves additional inherent and audit risks due

to the …

• shared responsibility

• greater technical complexity

• Consider SAS 70 or other relevant audit

standards

Step 4:

Identification of risks

and key controls

Step 4:

Risks and Key Controls

• Objectives:

– Mark out the audit

universe to be

subsequently analysed

– Establish what key

controls mitigate each

relevant risk

– Analyse the impact of

the assertions in the

financial statements

Step 4:

Risks and Expected (Key) Controls

• Identify risks that could lead to material

errors in the financial statements within the

main processes

• Establish expectations in terms of the typical

and expected controls for the risks identified

• Subdivide the controls into key controls and

others

Definition

Key Controls

Remark:

• Key controls are meant to guarantee the

reliability of the process result and the

financial data.

• Key controls constitute the backbone of the

control system and it is therefore essential

that they be checked by the auditor.

Step 4:

Compare Expected to Actual Controls

• The auditor should compare the expected

key controls to the actual key controls

existing in the process concerned

• All important controls related to applications

that have a direct influence on the process

result must be taken into consideration

Step 4:

Different Types of Controls

• Assess these types of

controls together !

Application ControlsIT-Dependent Manual

Controls

Manual Controls Automated Controls

(Purely) Manual Controls

IT General Controls

Application

dependent

controls

Source: Bernhard Hamberger, E&Y;


Step 4:


Source: Accounting Information Systems;

book written by Gelinas, Sutton, Oram

Step 4:


Source: Accounting Information Systems;

book written by Gelinas, Sutton, Oram

Risk/control matrix (example)

Step 4:


Risk/control matrix

Step 4:


Risk/control matrix (example)Source: Michel Huissoud,

Swiss Federal Audit Office

Step 5:

Walk-through

Step 5:

Walkthrough

• Objectives:

– Verify the auditor’s

understanding of the

processes, the relevant

risks and controls

– Thereby confirm the

analysis carried out

beforehand

Step 5:

Depth of Walkthrough

• The depth/degree of detail of a walkthrough

depends on whether the auditor intends to

rely on the existing control system or not:

– If the auditor intends to rely on the controls,

he/she will analyse the individual controls in

detail

– If the auditor does not intend to rely on the

effectiveness of the controls he/she will carry out

a less detailed walkthrough.

Step 5:

Verify During Walkthroughs

• With walkthroughs, the following are

systematically verified:

– Understanding of flows and processing

– The consistency and significance of existing

documentation and flow charts

– The accuracy and completeness of the

information regarding the relevant controls and

– The existence of relevant controls in the day-to-

day routine

Step 5:

Performing Walkthroughs

• The following questions must be taken into

consideration during the walkthrough:

– Who should be asked for explanations of details,

e.g. concerning the field of activity?

– Who/where do existing documents, reports,

flowcharts, screenshots come from?

– What control activity takes place for the

individual activities?

– Is the control implemented to prevent errors or to

discover them?

Step 5:

Performing Walkthroughs

• The following questions must be taken into

consideration during the walkthrough (cont.):

– How is the control implemented (automated or

manual) and how often?

– Is the automated control really implemented?

– What audit trail does the control leave?

Step 6:

Assessment of

control design

Step 6:

Evaluation of Control Design

• Objectives:

– Verify whether an

adequate control quality

can be achieved with

the lowest number of

controls

– Establish an adequate

audit strategy for the

evaluation of the

implementation of the

controls

Step 6:

Questions for Evaluating Design

• Questions to be posed concerning the

evaluation of control design

– Are the steps of the process and the related

controls arranged in a logical and sensible

order?

– Is the responsibility for the implementation of the

controls clearly established?

– Is it possible to implement the controls in a

correct and meaningful manner?

Step 6:



evaluation of control design (cont.):

– Are manual or hybrid controls replaced by

automatic controls wherever possible?

– Are upstream (preventative) controls used

instead of downstream (detective) controls

wherever possible?

– Do the controls comply with the requirements of

guidelines and procedural instructions?

Step 6:




– Are the inputs needed for the realisation of the

control available?

– Does the implementation of the control enable

the production of a control document that can be

checked?

– Are the controls carried out by a suitable,

qualified and trained operator?

Step 6:




– Are the controls carried out within an appropriate

period and sufficiently often?

– Can the design of the controls be implemented at

all within the organisational and financial limits

applicable?

Step 7:

Assessment of control

implementation

Step 7:

Testing Operating Effectiveness

• Objectives:

– Obtain an audit opinion

on the internal control

system

– Evaluate whether each

control

• functions as planned

• was actually and fully

implemented

• was appropriately

qualified

Step 7:

Testing Operating Effectiveness

• TOEs comprise the following steps:

– Selection of controls for audit, if the control

system as a whole is not to be audited;

– Choice of audit strategy;

– Choice of test procedure and sample size;

– Implementation of procedure-oriented/result-

oriented audit procedures;

– Evaluation of discrepancies and the materiality of

errors and weaknesses.

Step 7:

Some Testing Strategies

Audit strategies

• Test-of-one

• Direct testing

• Baselining/benchmarking

• Data analysis

Step 7:

Test of one

Test of one

In principle, programme controls need only

be tested once, after which, given a stable IT

environment and effective general IT controls

during the audit period, they can be

considered to be just as effective thereafter.

Step 7:

Direct Testing

Direct testing

The operation of the control is verified by

means of sampling or the analysis of

transaction data.

Step 7:

Baselining

Baselining/benchmarking:

The aim of this strategy is to reduce the

extent of substantive testing in subsequent

audit periods. To this end, the results of

testing application controls are carried

forward to subsequent audit periods. The

tests conducted during the first audit period

serve as a benchmark.

Step 7:

Data Analysis

Data analysis

The effectiveness of a control is verified by

means of computer-assisted data analysis,

whereby, ideally, all the relevant data in the

analysis are taken into consideration

Step 7:

Evaluating Control Implementation

• Evaluating the control implementation:

– Control frequency - the less frequently a

manual control is performed, the smaller the test

sample;

– Control significance - the more importance the

auditor attaches to an individual control when

forming his/her audit opinion, the more

thoroughly that control should be tested;

Step 7:


• Evaluating the control implementation (cont.):

– Soundness of control evidence – smaller

samples can be taken of controls that generate

direct evidence of effectiveness (traceability,

exhaustiveness, accuracy and time references)

than of controls offering no direct evidence;

– Relative importance of errors and anomalies

– the impact of these varies according to

materiality and the complexity and number of

processed transactions;

Step 7:


• Evaluating the control implementation (cont.):

– Management override – the likelihood that

managers will bypass or override a control;

– Frequency of changes to controls – a control’s

effectiveness may be very significantly affected

by changes to the control itself or to the control

environment.

Step 8:

Overall view and the

determination of the

audit opinion

Step 8:

Overall Assessment & Conclusion

• Objectives:

– Findings of each successive step in the audit are

evaluated for their impact on the financial

reporting

– Compilation of the findings of all preceding audit

steps

– Delivery of a final statement whether the internal

control system is able to provide reasonable

assurance that material errors are avoided in the

annual accounts

Step 8:


• There should be a global assessment of:

– the extent to which the audited application

makes an effective contribution to the business

process (control design and implementation);

– whether there are significant control weaknesses

in the application;

– the impact of control weaknesses on the

application’s effectiveness, the overall system

and the corresponding assertions in the

accounts;

Step 8:


• There should be a global assessment of

(cont.):

– whether the business process has any

compensatory controls to offset the impact of

control weaknesses in the application, and

whether there is a need for further testing of

controls and additional result-oriented audit

procedures.

Step 8:

Material Control Weaknesses

• Where control weaknesses in the application

are such as to materially compromise the

accuracy of the assertions in the annual

accounts, and where this risk is not offset by

compensatory controls, the impact of those

weaknesses on the year-end reporting must

be evaluated.

Step 8:

Material Control Weaknesses

• This evaluation is tackled from three angles:

1. Does the control weakness have an impact on

the financial statements?

2. Are there legal offences or violations of the

company statutes?

3. Does the weakness affect the audit opinion?

When evaluating the impact on the audit opinion,

the auditor considers whether substantive audit

procedures should be used to assess the

potential impact of ineffective controls.

Wrap up

Wrap-upIntegrated Approach to Auditing (IT) Applications

Wrap-upIntegrated Approach to Auditing (IT) Applications

• Nothing new, but a consistent approach to

integrated auditing

• Taught in Switzerland to all financial auditors

• Available in German, French, English (June)

• Be aware, that this approach (or the

brochure)

– does not replace audit know-how

– does not mean that you can neglect any

applicable audit standards or guidelines

Another Highly Valuable SourceIntegrated Approach to Auditing (IT) Applications

Examples of application dependent controls

IT Control Objectives for Sarbanes Oxley,

IT Governance Institute, ISBN 1-9332284-76-5

For More Information:

Peter R. Bitterli, CISA, CISM

Bitterli Consulting AG & ITACS Training AG

[email protected]

www.bitterli.com / www.itacs.com

Thank you!

Documents

Integrated Auditing of (IT) Application - Bitterli … · Integrated Auditing of IT Applications ... designed to provide reasonable assurance ... Financial Management module may not