Joint meeting of RDU IIA and ISACA November 11, 2008, Capitol Club, Raleigh, North Carolina Joint meeting of the RDU IIA and ISACA chapters November 11,

Joint meeting of RDU IIA and ISACANovember 11, 2008, Capitol Club, Raleigh, North Carolina

Joint meeting of the RDU IIA and ISACA chaptersNovember 11, 2008, Capitol Club, Raleigh, NC

Slide <#> Slide 1

ProcurementProcurement FraudFraudDetection and PreventionDetection and Prevention

November 11, 2008November 11, 2008

Mike BlakleyMike Blakley

Slide 2Joint meeting of the RDU IIA and ISACA chaptersNovember 11, 2008, Capitol Club, Raleigh, NC

Session objectives1. Current trends, techniques

and best practices2. Understand statistical basis

for analysis3. Procurement cards (p-

cards) 4. Understand use of Excel


Top Six IndicatorsThat you might have a fraud

• 6. System designed to do “three way match”, but only does two way

• 5. Procurement software system doesn’t do a match

• 4. When auditors ask to help them out, they point to the door

• 3. No procurement software system• 2. Procurement clerk drives a Porsche• 1. Clerk’s kids drive Porsches between

mountain home and beach home


Overview• Fraud patterns detectable

with digital analysis• Basis for digital analysis

approach• Usage examples• Using Excel


The Why and How

• Two brief examples• IIA Guidance Paper• Auditors “Top 10”• Process Overview• Who, What, Why, When &

Where

Objective 1


Example 1School Bus Transportation Fraud

• Supplier Kickback – School Bus parts

• $5 million• Jail sentences• Period of years

Objective 1


Regression Analysis

• Stepwise to find relationships– Forwards– Backwards

• Intervals– Confidence– Prediction

Objective 1


Data outliersObjective 1

• Sometimes an “out and out Liar”

• But how do you detect it?


Data Outliers

• Plot transportation costs vs. number of buses

• “Drill down” on costs– Preventive maintenance– Fuel– Inspection

Objective 1


Scatter plot with prediction and confidence intervals


Medicare HIV Infusion Costs

Objective 1

• CMS Report for 2005• South Florida - $2.2

Billion• Rest of the country

combined - $.1 Billion


Pareto ChartObjective 1

Medicare HIV Infusion Costs - 2005 ($Billions)data source: HHS CMS

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

County

Ann

ual M

edic

are

Cos

ts

Pct

Cum Pct


Guidance Paper• A proposed implementation

approach• “Managing the Business Risk of

Fraud: A Practical Guide” http://tinyurl.com/3ldfza

• Five Principles• Fraud Detection• Coordinated Investigation

Approach

Objective 1


Managing the Business Risk of Fraud: A Practical Guide

• IIA, AICPA and ACFE• Report issued 5/2008• Section 5 – Fraud

Detection

Objective 1


Section 5 – Fraud Detection

• Detective Controls• Process Controls• Anonymous Reporting• Internal Auditing• Proactive Fraud

Detection

Objective 1


Proactive Fraud Detection

• Data Analysis to identify:– Anomalies– Trends– Risk indicators

Objective 1


Specific Examples Cited

• Journal entries – suspicious transactions

• Identification of relationships• Benford’s Law• Continuous monitoring

Objective 1


Data Analysis enhances ability to detect fraud

• Identify hidden relationships• Identify suspicious transactions• Assess effectiveness of internal

controls• Monitor fraud threats• Analyze millions of transactions

Objective 1


Peeling the Onion

Population as Whole

Possible Error Conditions

Fraud Items

Objective 1c


Fraud Pattern Detection

Market Basket

Stratification

Trend Line

Holiday

Day of Week

Duplicates

Univariate

Gaps

Benford’s Law

Round Numbers

Target Group

Objective 1d


Who Uses Analytics

• Traditionally, IT specialists• With appropriate tools,

audit generalists (CAATs)• Growing trend of business

analytics• Essential component of

continuous monitoring

Objective 1e


Analytics – what is it?• Using software to:

– Classify– Quantify– Compare

• Both numeric and non-numeric data

Objective 1e


How - Assessing fraud risk • Basis is quantification• Software can do the “leg work”• Statistical measures of

difference– Chi square– Kolmogorov-Smirnov– D-statistic

• Specific approaches

Objective 1e


Why - Advantages• Automated process• Handle large data populations• Objective, quantifiable metrics• Can be part of continuous monitoring• Can produce useful business analytics• 100% testing is possible • Quantify risk• Repeatable process

Objective 1e


Why - Disadvantages

• Costly (time and software costs)

• Learning curve• Requires specialized

knowledge

Objective 1e


When to Use Analytics

• Traditional – intermittent (one off)

• Trend is to use it as often as possible

• Continuous monitoring• Scheduled processing

Objective 1e


Where Is It Applicable?

• Any organization with data in digital format, and especially if:– Volumes are large– Data structures are complex– Potential for fraud exists

Objective 1e


Objective 1 Summarized

• Two brief examples• IIA Guidance Paper• “Top 10” Metrics• Process Overview

Objective 1


Objective 1 - Summarized1. Understand why and how 2. Understand statistical basis for

quantifying differences3. Identify ten general tools and

techniques

Next is the basis …


Basis for Pattern Detection

• Analytical review• Isolate the

“significant few” • Detection of errors• Quantified approach

Objective 2


Trapping anomaliesObjective 3Objective 2


Understanding the Basis

• Quantified Approach• Population vs. Groups• Measuring the Difference• Stat 101 – Counts, Totals,

Chi Square and K-S• The metrics used

Objective 2


Quantified Approach

• Based on measureable differences

• Population vs. Group• “Shotgun” technique

Objective 2a


Detection of Fraud Characteristics

• Something is different than expected

Objective 2a


Fraud patterns• Common theme –

“something is different”• Groups• Group pattern is different

than overall population

Objective 2b


Measurement Basis

•Transaction counts

•Transaction amounts

Objective 2c


How is digital analysis done?• Comparison of group with

population as a whole• Can be based on either counts or

amounts• Difference is measured• Groups can then be ranked using

a selected measure• High difference = possible

error/fraud

Objective 2d


Histograms• Attributes tallied and categorized

into “bins”• Counts or sums of amounts

Objective 2d


Two histograms obtained• Population and group

Population

0

100

200

300

400

500

600

700

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Group

01020304050607080

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Objective 2d


Compute Cumulative Amount for each

Count by Month

0

10

20

30

40

50

60

70

80

Month

Cou

nt

Cum Pct

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

Jan-

07

Mar

-07

May

-07

Jul-0

7

Sep-0

7

Nov-0

7

Objective 2d


Are the histograms different?

• Two statistical measures of difference

• Chi Squared (counts)• K-S (distribution)• Both yield a difference

metric

Objective 2d


Chi Squared

• Classic test on data in a table

• Answers the question – are the rows/columns different

• Some limitations on when it can be applied

Objective 2d


Chi Squared

• Table of Counts• Degrees of Freedom• Chi Squared Value• P-statistic• Computationally intensive

Objective 2d


Kolmogorov-Smirnov

• Two Russian mathematicians

• Comparison of distributions• Metric is the “d-statistic”

Objective 2d


How is K-S test done?

• Four step process1. For each cluster element

determine percentage2. Then calculate cumulative

percentage3. Compare the differences in

cumulative percentages4. Identify the largest difference

Objective 2d


Kolmogorov-SmirnovObjective 2d - KS


Classification by metrics• Stratification• Day of week• Happens on holiday• Round numbers• Variability• Benford’s Law• Trend lines• Relationships (market basket)• Gaps• Duplicates

Objective 2e


Fraud Pattern Detection

Market Basket

Stratification

Trend Line

Holiday

Day of Week

Duplicates

Univariate

Gaps

Benford’s Law

Round Numbers

Target Group

Objective 3


What can be detected• Made up numbers

– e.g. falsified inventory counts, tax return schedules

Objective 2


Benford’s Law using Excel• Basic formula is “=log(1+(1/N))”• Workbook with formulae available

at http://tinyurl.com/4vmcfs

• Obtain leading digits using “Left” function, e.g. left(Cell,1)

Objective 2


Made up numbers• Benford’s Law• Check Chi Square and d-statistic• First 1,2,3 digits• Last 1,2 digits• Second digit• Sources for more info


How is it done?• Decide type of test – (first 1-3 digits,

last 1-2 digit etc)• For each group, count number of

observations for each digit pattern• Prepare histogram• Based on total count, compute

expected values• For the group, compute Chi Square

and d-stat• Sort descending by metric (chi

square/d-stat)

Objective 2


Invoice Amounts tested with Benford’s law - Example Results

During tests of invoices by store, two stores, 324 and 563 have significantly more differences than any other store as measured by Benford’s Law.

Store Hi Digit Chi Sq D-stat

324 79 5,234 0.9802

563 89 4,735 0.97023

432 23 476 0.321

217 74 312 0.2189

Objective 2


Next Metric1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Objective 2


Duplicates

Why is there more than one?

Same, Same, Same, and

Same, Same, Different

Objective 2


Two types of (related) tests• Same items – same vendor,

same invoice number, same invoice date, same amount

• Different items – same employee name, same city, different social security number

Objective 2


Duplicate Payments

• High payback area

•“Fuzzy” logic• Overriding

software controls

Objective 2


Fuzzy matching with software

• Levenshtein distance• Soundex• “Like” clause in SQL• Regular expression

testing in SQL• Vendor/employee

situations

Russian physicist

Objective 2


How is it done?

• First, sort file in sequence for testing

• Compare items in consecutive rows

• Extract exceptions for follow-up

Objective 2


Possible Duplicates - Example Results

Five invoices may be duplicates.

VendorInvoice

DateInvoice Amount Count

10245 6/15/2007 3,544.78 4

10245 8/31/2007 2,010.37 2

17546 2/12/2007 1,500.00 2

Objective 2


Next Metric1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Objective 2


Holiday Date Testing• Red Flag indicator

Objective 2


Typical audit areas

• Invoices• Receiving reports• Purchase orders

Objective 2


Federal Holidays

• Established by Law• Ten dates• Specific date (unless

weekend), OR• Floating holiday

Objective 2


Understanding the Basis

• Quantified Approach• Population vs. Groups• Measuring the Difference• Stat 101 – Counts, Totals, Chi

Square and K-S• The metrics used

Objective 2


Objective 2 - Summarized

1. Understand why and how 2. Understand statistical basis for

quantifying differences3. Procurement cards4. Understand examples done

using Excel

Next up: p-cards …

Objective 2


Testing Procurement Card Transactions

1. Understand Merchant Charge Codes (MCC)2. Understand common policies3. Test procurement card transactions contained on

worksheets using VBA4. Ability to test procurement card transactions in a file

using VBA5. Perform an audit of procurement card transactions in a

more efficient and effective manner using the concepts and techniques presented

Objective 3


Audit Benefits(How this test supports the audit) • Test compliance with policy on an

account by account basis• Test compliance with policies on

account limits• Enable 100% testing of transactions• Audit process which can be tailored for

policy changes• Repeatable audit process

Objective 3


MCC Structure

• Major Categories• Airlines 30XX – 32XX• Car Rental 33XX, 34XX• Hotels 35XX – 37XX• All Other

Objective 3


Policy Structure

• Prohibited Codes• Codes allowed with a

monthly limit• Codes allowed without limit• Overall card limit

Objective 3


Summary and Wrap Up

1. Understand Merchant Charge Codes (MCC)2. Understand common policies3. Test procurement card transactions contained

on worksheets using VBA4. Ability to test procurement card transactions in

a file using VBA5. Perform an audit of procurement card

transactions in a more efficient and effective manner using the concepts and techniques presented

Objective 3




quantifying differences3. Procurement cards4. Understand examples done

using Excel

Next up: Excel …


Use of Excel• Built-in functions• Add-ins• Macros• Database access

Objective 4


Excel – Univariate statistics• Work with Ranges• =sum, =average, =stdevp• =largest(Range,1),

=smallest(Range,1)• =min, =max, =count• Tools | Data Analysis |

Descriptive Statistics

Objective 4


Excel Histograms

• Tools | Data Analysis | Histogram

• Bin Range• Data Range

Objective 4


Excel Gaps testing• Sort by sequential

value• =if(thiscell-lastcell <>

1,thiscell-lastcell,0)• Copy/paste special• Sort

Objective 4


Detecting duplicates with Excel

• Sort by sort values• =if testing• =if(=and(thiscell=l

astcell, etc.))

Objective 4


Performing audit tests with macros

• Repeatable process• Audit standardization• Learning curve• Streamlining of tests• Examples -

http://tinyurl.com/576tp8

Objective 4


Use of Excel

• Built-in functions• Add-ins• Macros

Objective 4




quantifying differences3. Identify ten general tools and

techniques4. Understand examples done using

Excel


Questions?


Links for more information• Kolmogorov-Smirnov • http://tinyurl.com/y49sec• Benford’s Law

http://tinyurl.com/3qapzu• Chi Square tests

http://tinyurl.com/43nkdh• Continuous monitoring

http://tinyurl.com/3pltdl


Excel macros used in auditing

• Excel as an audit software http://tinyurl.com/6h3ye7

• Selected macros - http://tinyurl.com/576tp8

• Spreadsheets forever - http://tinyurl.com/5ppl7t


Contact info

• E-mail: [email protected]

• Web: http://ezrstats.com

Documents

Joint meeting of RDU IIA and ISACA November 11, 2008, Capitol Club, Raleigh, North Carolina Joint meeting of the RDU IIA and ISACA chapters November 11,