QUALITY ASSURANCE

CHAPTER 24

GROUP 5 :Amelia

ZulaikhaDhani Arinta A JKalyana Darma

LeksonoMahardian

HakimMuhammad

FirmanRahayu Iswara

Ummasyita Nurul

Introduction

In 1978, The Institute of Internal Auditor (IIA) publish the Standards for the Professional Practice of Internal Auditing (Standards)

The Standard on Quality Assurance

“The chief audit executive should establish and maintain a quality assurance program to evaluate the operations of the internal auditing activity”

Element of Quality Assurance Program:•Supervision•Internal Reviews•External Reviews

The Standard on Quality Assurance

The current Standard 1310, “Quality Program Assessments,” States:

“The Internal audit activity should adopt a process to monitor and assess the overall effectiveness of the quality program. The process should include both internal and external assessments”

The Standard augmented by Practice Advisory 1310-1

Practice Advisory

The Practice Advisory specifically suggest that internal audit should be perceived by stakeholders as adding value and improving the organization’s operations.

The assessment process of quality programs should include recommendations, if appropriate relative to:•Compliance with the Standard and Code of Ethics•Adequacy of the activity’s charter, goals, objections, policies and procedures•Contributions to risk management , governance, and control processes•Compliance with government laws, regulations, and industry standards•Effectiveness of operations•Addition to value and improvement of operations

Evolution of Quality Assurance Reviews

• Reviews are designed to be independent analyses that identify whether :1. The audits are meeting the needs of those who depend

on them.2. The audit operations are being performed correctly.3. The auditing can be done better. Whether more should

be done.4. Maximum value is being received for each internal audit

dollar of expense.5. Internal audits meet current professional standards.

Internal Reviews

• The internal review is to give the chief audit executive :1. Assurance that the activity is in compliance with the

Standards2. Confidence that activities not requiring the executive’s

personal involvement comply with policy and procedures3. Identification as to the level of audit effectiveness and

efficiency4. Information for improving the operation5. Assistance in preparing for external reviews

External Reviews• The purpose : to provide an independent evaluation for management

and the board audit committee.• 8 internal auditing program activities :

1. Planning2. Professional proficiency3. Risk analysis and audit planning4. Analysis of audit plan accomplishment5. Individual audit review6. Special project review7. Audit policies and procedures8. Computer audit scope

• Some matters that are touched upon in the Manual need special attention :1. The structure of the internal auditing activity2. The audit process and human relations3. Management of the audit function4. Evaluation of audit result

Current Quality Control in Internal Auditing

Steps Times Mentioned

Development of mission and vision statement and establishment of internal auditing activity objective

47

Establishment and implementation of performance measures for various stages of the internal auditing process

43

Identification of customers of internal auditing department 38

Development and implementation of internal auditing customer satisfaction surveys and feedback systems

33

Benchmarking with other internal auditing departments 31

Introspective self analysis 30

TQM training and education of the internal auditing staff 15

The importance of optimum development of internal auditing performance measures :

Phase Yes

Planning 48 58%

Field Work 52 63%

Audit Report 62 76%

Productivity in Internal Auditing

• Recent articles on internal audit productivity emphasize such aspects as :1. The total amount of dollars saved or collections made as

a result of total audit activity2. Number of audits completed3. Number of audit findings4. Number of audits completed within ± 5 % of budget5. Etc.

Characteristics and classification of the findings are a far more important measures

Evaluating Audit Productivity

Productivity in service operations such as internal auditing is difficult to measure because :

• Unable to identify and quantify the outcome• Difficult to value the work• Responsible to the output• Have much discretion

3 elements of quality assurance program

Who can audit the auditors?

• Peer group within the enterprise• The enterprise’s external auditors• External auditors from another accounting firm• Reciprocal evaluations between audit groups of

different enterprise• Qualified consultant• The IIA Quality Auditing service

Supervisors

Supervisor should :• Discuss the trust and scope of the audit before

preliminary survey• Conduct review regularly• Review audit working paper and provide evidence of

such review• Monitor budget and schedule

Internal reviews

• Provide : quality assurance & training• Forms : verifications, internal reviews, &client evaluation• Frequency : adequacy of the supervision of audit projects &

amount of adverse feedback from clients• Practice advisory suggest:1. ongoing reviews using measures2. periodic reviews performed by members of internal audit3. quality of performance & appropriate action to achieve

improvements.

Verifications

• Chief audit executives or audit managers• Verifications system to detect mechanical errors and faulty

judgments• Staff auditor: trace the working paper to make sure it is

supported & documented• All calculation should be recalculated• Staff reviewer: sign a quality reviewer checklist as evidence of

independent verification

Internal Review Program Appraisals of how well auditors & supervisors have complied, require the

attention of senior staff auditor / supervisors Salutary results: 1. information supplied to chief audit executive, information

of how well procedures are followed & how well the audit work & audit reports are documented. 2. value to the external auditors.

Internal reviewers of internal audits should be carried with formality & discipline of any audit examination, example:

1. Reviewed with a budget & a schedule2. Prepared that sets forth the steps the evaluator will take3. Acceptable sample of audit projects should be selected that’s representative

of the activity’s production.4. Evaluator should discuss any deficiencies with the auditors & supervisors of

the audit projects reviewed. 5. Evaluator should prepare working papers documenting the internal review6. Evaluator should also prepare formal report on the result of the review Should be approved by chief executive auditors

Internal Review Program

How well the auditors planned the audit work?How well the auditor collected, analyzed, &

documented information to support audit results?How well the results of the audit work were

reported?How well the auditor followed up to see that

appropriate action was taken on reported findings

Quality Circles

Methods improving quality & auditing productivity

Peer review study as to structure & contentStudies the operation, makes

recommendations, & frequently has the authority to implement them

External ReviewsCarried out by the peers of those to be reviewed:

doctors / academicians usually called for by the committee, CEO, Executive

VP or controller. Qualifications of the external reviewers:1. Independence2. Integrity & objectivity3. Competence

External Reviews

Scope of assessments include :1. Compliance with standards & code of ethics2. Compliance with charter, plans, policies & procedures, & legal

requirements3. Consideration of the organization’s expectations4. Degree of integration of internal audit activity into the total

organization’s fabric5. Mix of knowledge, experience, & discipline of internal audit staff6. Degree of value added to the organization resulting from the

internal audit activity.

Metodologi Penelaahan

Standar mengungkapkan bahwa aktivitas audit internal dewasa ini bekerja di lingkungan yang berbeda satu sama lain, yang selanjutnya menentukan metodologi dan lingkup dari pekerjaan audit internal yang dilaksanakan.

Beberapa aktivitas audit internal memiliki piagam audit dengan lingkup luas. Piagam audit aktivitas audit internal tidak dibatasi oleh pernyataan kebijakan apa pun untuk mematuhi standar secara penuh.

• Auditor internal diatur oleh lingkungan organisasi masing-masing. Oleh karena itu, mereka mungkin tidak dapat memenuhi seluruh konsep yang ditetapkan dalam Standar dan mungkin dapat dianggap tidak melaksanakan praktik profesional dari audit internal seperti yang dimaksud oleh Standar.

• Di lain pihak, peer review yang tidak dibatasi mungkin dapat memberitahukan kepada komite audit dewan dan manajemen senior kenyataan bahwa dibutuhkan perluasan peran auditor internal sebelum mereka dapat dianggap telah memenuhi standar profesionalnya.

• Berikut ini disajikan pembahasan mengenai penelaahan eksternal yang tercakup dalam Manual dengan lebih rinci.

Metodologi Penelaahan

Penelaahan eksternal pada umumnya diputuskan oleh komite audit dewan, pihak yang menerima laporan direktur audit, dan direktur audit itu sendiri.

Dimulai dengan pemberian informasi kepada pihak-pihak tersebut bahwa tujuan dari diadakannya penelaahan ini adalah:

• Untuk mengevaluasi kepatuhan terhadap standar dan kebijakan serta prosedur perusahaan.

• Untuk menilai kualitas dari operasi aktivitas.• Untuk memberikan rekomendasi perbaikan.

Sebuah surat penugasan sebaiknya dibuat dengan mencantumkan:• pengidentifikasian lingkup dari penelaahan; • tanggung jawab aktivitas audit di dalam penelaahan (sebuah evaluasi studi yang

dikerjakan sendiri); • tanggung jawab tim penelaah; • estimasi biaya;• klausul keamanan untuk melindungi tim penelaah dari kewajiban lebih lanjut• nama-nama dari anggota tim penelaah

Persiapan

Tim penelaah sebaiknya dipilih berdasarkan independensinya dan objektivitasnya, pengalaman, keluangan waktu, dan keahlian spesifik lain yang diperlukan (teknologi informasi, teknik industri, akuntansi, dll)

Tim penelaah selanjutnya melakukan survei terhadap klien dari aktivitas audit. (wawancara atau dengan memberikan kuesioner yang memuat kondisi-kondisi utama seperti: kerahasiaan, pemahaman, dan sebuah sampel perwakilan).Manual memberikan sebuah contoh kuesioner beserta surat pengantarnya. Surat pengantar dan kuesioner dibuat dengan:

– Menjelaskan proses pelaksanaan survei dan keuntungannya bagi responden.– Menjamin kerahasiaan dari para responden.– Singkat.– Berkonsentrasi pada area-area utama.– Berisi judul yang mengungkapkan suatu uraian yang logis– Menghubungkan tiap pertanyaan hanya pada satu permasalahan

Tim penelaah sebaiknya melakukan wawancara dengan orang-orang yang tidak mungkin tidak layak untuk menerima kuesioner; misalnya anggota komite audit dan manajemen senior

Jawaban-jawaban survei kemudian dianalisis dengan hati-hati. Tersedia berbagai paket peranti lunak yang dapat melakukan analisis statistik atas jawaban-jawaban tersebut.

Aktivitas audit hendaknya diminta untuk menyiapkan sebuah laporan self-study yang dapat membantu memfasilitasi penelaahan ini. Laporan ini sebaiknya memberikan informasi mengenai:• Dokumen-dokumen dasar (anggaran operasi, manual kebijakan, dan sejenisnya)• Data statistik mengenai organisasi, aktivitas, dan stafnya.• Diskusi mengenai aktivitas- pernyataan misi, sasarannya, lingkup audit, karyawan,

dan afiliasi-afiliasi profesional.• Metode perencanaan dan penganggaran.• Hubungan dengan manajemen eksekutif dan dewan komisaris.• Hubungan dengan auditor eksternal dan badan-badan pengawas lainnya.• Bagaimana aktivitas melaksanakan audit internal• keahlian-keahlian khusus yang dimiliki aktivitas• Sifat dari kontrol kualitas yang diterapkan oleh direktur audit internal.

Ketua tim akan membuat kunjungan awal membahas usulan penelaahan, self-study, orang-orang yang akan diwawancarai, waktu pelaksanaan pekerjaan lapangan, dan pemilihan kantor-kantor cabang yang hendak dikunjungi.

Tim meminta dan menelaah laporan self-study, informasi survei klien, dan dokumen pendukung.

Tim mewawancarai anggota staf audit dan pihak-pihak lain di dalam organisasi seperti manajer keuangan, kontroler, bendaharawan, manajemen operasi, auditor eksternal, komite audit, dan orang yang menerima laporan direktur audir internal

Tim menelaah keahlian audit khusus. (seperti ilmu teknik, ilmu aktuaria, ilmu fisika,dll)

Tim memeriksa sampel perwakilan dari kertas kerja dan pelaporan proyek audit serta menelaah praktik-praktik administrasi (penelaahan sebaiknya diarahkan sesuai dengan persyaratan dalam standar IIA).

Manual juga memberikan langkah-langkah audit yang rinci untuk seluruh tahapan di dalam pekerjaan lapangan peer review

Pekerjaan Lapangan

Tim penelaah sebaiknya menyiapkan sebuah laporan tertulis pada saat menyimpulkan hasil pemeriksaannya.

Draf laporan hendaknya dikoordinasikan dengan anggota tim dan diberikan kepada direktur audit untuk ditelaah.

Laporan final sebaiknya dialamatkan kepada orang atau kelompok yang meminta dilaksanakannya penelaahan, dengan salinan diberikan kepada direktur audit dan pihak yang menerima laporan direktur audit secara administratif.

Direktur audit kemudian hendaknya memberikan jawaban tertulis atas laporan audit tim.

Tindak lanjut dari setiap tindakan perbaikan sebaiknya dilaksanakan sebagai suatu bukti yang objektif bahwa setiap kelemahan yang dilaporkan telah benar-benar diperbaiki. Tindak lanjut umumnya akan dilaksanakan pada waktu dilakukannya penelaahan berikutnya.

Manual memberikan sebuah contoh laporan dari hasil penelaahan eksternal

Pelaporan

Contoh laporan yang terdapat dalam manual memberikan beberapa indikasi mengenai beberapa jenis temuan yang dapat diidentifikasikan oleh peer review. Contohnya:

• Auditor internal melaksanakan beberapa fungsi lini operasional.• Dibutuhkan sebuah program pelatihan dan pendidikan berkelanjutan yang formal.• 30% waktu audit dihabiskan di bagian akuntansi. Wawancara dengan klien, ditambah

dengan observasi independen, mendukung kesimpulan mengenai perlunya penambahan waktu pada area-area operasional di luar fungsi akuntansi.

• Tidak adanya sistem pelaporan proyek formal yang digunakan untuk mencatat waktu yang dianggarkan dan digunakan dalam setiap audit.

• Pada beberapa kasus, terdapat kurangnya pekerjaan yang dilakukan dalam melakukan analisis atas kontrol keuangan pada saat audit aktivitas keuangan.

• Dalam 20% proyek audit yang ditelaah, terdapat perbedaan yang signifikan antara pekerjaan yang diprogramkan dengan yang dicapai. Tidak terdapat tanda tangan persetujuan oleh supervisor untuk pendokumentasian perubahan program yang terjadi.

• Kertas kerja yang ada tidak menunjukkan bukti penelaahan oleh supervisor.• Manual kebijakan audit tidak memuat instruksi mengenai perencanaan audit, supervisi,

penilaian kinerja, kontrol proyek, dan pendidikan berkelanjutan.

Peer Review

Peer Review of Multi-site Operations

• The composition of the quality review team as consisting of “seasoned auditors with special skills relating to the various areas audited”

• The peer review is conducted like an audit and covers a particular program and particular audits of the multi-site internal auditing activities

Reviews by Other Professional Organization

• The selection of an outside organizations be made so as to eliminate any benefits that would accrue to the firm as a result of advance report.

• Advantage: can make comparisons with other organizations

Clients Evaluations

• The chief audit should request the manager of the audited organization to evaluate the audit functions.

• The chief audit executive or a senior manager should:– Review the evaluation– Call for a written response

Total Quality Management Implementation

• Series of steps:– Initial quality assessment– Top-level audit management awareness– Formation of a quality council– Fostering team work– Development of prototypes– Celebration of success– Organizational implementation– Annual quality review– Benchmarking

Initial Quality Assessment

Top-level Audit Management Awareness

Must understand that quality management is a philosophy or an approach to management –

not a program

Formation of a Quality Council

Fostering Teamwork

Development Prototypes

Celebrations of Success

Organizational ImplementationSuccessful method

should be implemented by all units

Annual Quality Review

BenchmarkingBenchmarking

This methodology comprises the mechanics of measuring and evaluating those aspects of an operation that are considered vital and essential to a quality operation.

Objective : to make comparisons with other organizations, with segments within organization, and with standards that have been set by management and the quality councils.

Measurements that are used are referred to as benchmark metrics because of their quantifiable nature.

These metrics are, example:»Quantity (units per; calls per; trips per; hours per; pages per)»Quality (defects per; retests per; failures per; complaints per)»Cost (costs per)

The setting of benchmark follows a normal analytical process. The council must:1.Decide what is to be measured 2.Determine the sources of the benchmark, internally and externally3.Establish data collection methods4.Set up analytical techniques together with methods to determine causes of deviations from goals or from top comparative units5.Determine methods to resolve problems causing deviations6.Follow-up to ensure progress in implementing corrective methods

The TQM Program of Southern California Edison Company

The TQM Program of Southern California Edison Company

The firm decided in 1993 to develop a TQM program and after some evolutionary activities finally in 1995 set up 2 groups of audit teams.

Group 2:»TQM steering committee»Team skills and education»Special projects»TQM measuring and reporting

Group 1:»Audit administration»Contact and program»Corporate organization»Customer Service»IT»New business»Nuclear and power production»Power contract and joint project

Group 1 (Natural Work)◌Responsible to normal audit, for continuous improvement of processes, products, and client satisfaction◌Membership : automatic assignment of people

Group 2 (The voluntary TQM teams)◌Cross functional teams, membership: voluntary◌Responsible for implementing TQM strategies

The teams developed an 8 steps development implementation management cycle actually includes the activities that the teams should perform in each phase of the cycle and is a complete outline of TQM engagement.

The audit organization then developed a matrix establishing :◌Primary Clients◌Client and corporate expectations◌Audit department TQM strategy

Series of tools to be used in the quality-improvement evaluation:1.Benchmarking program to collect information on audit quality and productivity from similar type organizations2.Client management satisfaction survey3.Annual self-assessment questionnaire4.Internal peer review checklist5.Audit draft report quality evaluation form6.Audit activity performance metrices7.Semi annual and annual reports on the status of the TQM program8.TQM newsletters

Quality Concern of Management and BODsQuality Concern of Management and BODs

The efficiency of internal control system cannot be evaluated without considering the organization’s organizational structure, the caliber of its employees, the strength of its audit committee, the effectiveness of its internal audit operations, and a host of other factors that, while not part of internal control system itself, have an impact on the functioning of the system.

The effectiveness of any internal audit operation can best be demonstrated by adequate quality assurance programs carried out by and for the internal audit organization.

ISO 9000 – A ChallengeISO 9000 – A Challenge

ISO 9000 series comprises 5 individual but related standards on quality management and 2 in auditing and measuring (ISO 10000 series).The series comprises:๑ISO 9001 -- covering design, manufacturing, installation, and service systems๑ISO 9002 -- covering production and installation ๑ISO 9003 -- covering final product inspection and testing๑ISO 9004 -- providing guidelines for producing the organization’s own quality system๑ISO 10011 -- containing guidelines for auditing quality systems๑ISO 10012 -- containing quality assurance requirements for measuring equipment

Q & A

Kelompok 2

•Cara menyusun quality assurance program ?

1. membuat standar

2. membuat kebijakan dan prosedur

3. Membuat deskripsi quality program

4. membentuk quality committee

5. Mengimplementasikan rencana tindakan korektif

• Kelompok 6Sejauh mana auditor internal bertanggungjawab

terhadap kesalahan yang terjadi pada saat pembuatan quality assurance review?

-> tidak ada kesalahan pada review, karena disesuaikan dengan standar

Apa bedanya quality assurance dan quality control?

-> quality assurance berhubungan dg proses. Quality control berhubungan dengan output produk tsb.

Q & A

• Kelompok 4 Selain menggunakan pendekatan TQM, juga ada balance

scorecard, mungkin ga pake balance scorecard? bagaimana contoh penerapan keempat perspektifnya?

Kedua tools mempunyai fungsi yang berbeda. BSC untuk evaluasi kinerja perusahaan, TCM untuk kualitas manajemen

Untuk konteks review by other profesional, seberapa pentingkah evaluasi dari klien dalam proses audit? Apakah evaluasi dari klien akan mempengaruhi

evaluasi klien tidak bisa mempengaruhi review by other profesional

Q & A

Kelompok 1Bagaimana cara mengatasi perbedaan hasil internal review dan eksternal review?

internal = staf auditor senior/supervisor, eksternal = pihak profesional/akademisi

Apakah klien evaluation dapat menjadi faktor penilaian efektif untuk menilai quality audit report?

Klien evaluation bukan satu2nya cara untuk mengevaluasi quality audit report. Proporsi keefektivitasan klien evaluation didalam quality audit report itu lebih kecil dibandingkan dengan peer review

Q & A

Kelompok 3Maksud konsep benchmarking yang lebih formal dan lebih disiplin?

karena semakin banyaknya perusahaan dalam industri, makin banyak standar, dan bahan benchmarking (industri pembanding) semakin bertambah banyak pilih best practice sebagai benchmark.

Q & A