1

PROCESS MINING: A NEW TECHNIQUE FOR EDP-AUDITING?

Doorga, Prashand, Erasmus University Rotterdam, The Netherlands, 253594pd@student.eur.nl

Janglie, Arun, Erasmus University Rotterdam, The Netherlands, 265882aj@student.eur.nl

Abstract Process mining is a new, growing discipline in the world of business and especially in the world of business process analysis. The goal of process mining is to extract an explicit process model from event logs recorded by an Enterprise Information System.

The EDP-auditor is has to determine whether evaluated information systems are safeguarding assets, maintaining data integrity, and operating effectively and efficiently to achieve the organization’s goals or objectives.

In this paper the topic of process mining is put into the context of EDP-auditing to see whether it can be a helpful tool for the EDP-auditor in his work field.

Key words: Process mining, EDP auditing

2

1 INTRODUCTION

Processes are the main engine of any organization. The business process is a fundamental concept in any enterprise and to be able to evaluate these processes it is important to have an (schematic) overview of how the business processes (the workflows) within the enterprise are. Information technology is being widely used in organizations nowadays. In the recent years Workflow Management Systems have offered generic modelling and enactment capabilities for structured business processes. Also other software such as ERP systems, CRM systems have made use of explicit process models. In these software systems still little attention has been paid to the monitoring and improvement of the business process models. Business Process Analysis is the term used for evaluating the business processes to help an organization improve its processes and thus how it conducts its functions and activities. Business Activity Monitoring is one of the emerging areas in BPA. The goal of BAM tools is to use data logged by the information system to diagnose the operational processes. Information systems in organizations make a great deal of transaction every single day. These transactions are all recorded in event logs and these event logs are at the basis of process mining. Process mining can be seen as a technology used to monitor the operation business processes. The goal of process mining is to extract information from event logs recorded by an information system and given such a log process mining tools will create a process model consistent with the observed dynamic behaviour. The rich data resources lying around in the transaction and workflow application software can be turned into vital knowledge about business operations.

When computer technology came into accounting systems the way of storing, retrieving and controlling of data changed. The first use of a computerized accounting system was at General Electric in 1954. During 1954 to the mid of 1960s, the audit profession was still doing its work with less usage of computer capacity. At this time only big mainframe computers were used and a few people had the skills and abilities for programming.

After this period, this changed because the introduction of new, smaller and less expensive machines the computer had more possibilities for an auditor to use. The use of computer in businesses increased very much and with it came the need of auditors to become familiar with EDP concepts in business. With the increasing usage of computer the rise of different types of accounting systems had raised.

The industry realized that they needed to develop Audit software and this generalized audit software (GAS) was developed. In 1968 the American Institute of Certified Public Accountants (AICPA) had the Big Eight accounting firms participate in the development of EDP auditing. The result of this that auditing and EDP was combined and that Auditing & EDP was released. [1]

1.1 Research

With process mining being a new, growing discipline the question is whether it can have applications in other areas of business than Business Process Management. For the field of EDP-auditing no research has been done on the topic of process mining as a tool for the EDP-auditor so far.

To be able to research whether process mining can be of any use to the EDP-auditor the following research question is formulated:

‘Are there EDP auditing applications for BPM process mining?’

To answer this question first a set of sub questions needed to be formulated and answered. The sub questions are:

3

‘What is BPM process mining?’

‘What types of analysis are possible with process mining?’

‘What is EDP-auditing?’

‘What is the work field of the EDP-auditor?’

‘What is the relationship between EDP-auditing and process mining?’

This paper consists of chapters giving answers to these sub questions. With the knowledge gained by answering these sub questions an attempt is made to answer the main research question of this paper and also an attempt is made to give insight in whether process mining can be of any help to the EDP-auditor.

2 PROCESS MINING

The technique of process mining can be used to retrieve information from the event logs which have data stored from information systems. The audit trails of workflow management system or the transaction logs of an ERP system can be used for this technique. Process mining can then extract model which describe processes, organization and products. It can also be used to monitor deviations through the evaluation of the actual events occurred with the predefined models or the business rules.

BAM (Business Activity Monitoring), BOM (Business Operations Management), BPI (Business Process Intelligence), and data/workflow mining are all work fields closely related to Process mining. Unlike classical data mining techniques the focus is on processes and questions that transcend the simple performance-related queries supported by tools [2].

When having a look at the scope of process mining the figure 1 can help to clarify. Figure 1 gives an overview of the role of information systems and its interaction with the real world (e.g. business processes, organizations, people, machines), the role of models and its influence on the real world, the effect models can have on information systems and the different types of analysis that can be done by process mining technology.

4

Figure 1: Process mining scope [2]

Any information system interacts with some physical environment (the real world), otherwise it serves no purpose. Such an information system supports and/or controls processes that are taking place in the real world. Information systems also record events, such as messages and transactions, taking place inside and outside the system [3, 4]. The recorded events are stored in event logs (also referred to as transaction log or audit trail) and information systems nowadays store a huge amount of data in these event logs. The stored data provide very detailed information about the activities that have been executed. Such an event log registers the start and/or completion of activities. Every event refers to a case (i.e. process instance) and an activity, and, in most systems, also a timestamp, a performer, and some addition data. The event logs are the starting point for process mining. And through process mining different perspectives of analysis can be distinguished: (i) the process perspective, (ii) the organizational perspective and (iii) the case perspective. The process perspective focuses on the control-flow, i.e. the ordering of activities. The goal of this perspective is to find the best path of all possible paths. The organizational perspective focuses on the originator field, i.e. which performers are involved and how are they related. The goal of this perspective is to structure the organization by classifying people in terms of roles and organizational units or to show the relations between individuals. The case perspective focuses on properties of cases [5].

Models also play an important role as is to be seen in figure 1. The model is an abstract representation of the real world in which important aspects of that real world are represented. With these models analyses and experiments can be performed to learn about the real world. The knowledge gained from these analyses and experiments can then be used to change/improve certain aspects of the real world. In figure 1 the different types of analyses (process discovery, conformance, and extension) are all

5

process mining techniques. These techniques do analysis of run-time behavior and this is only possible if events are recorded.

2.1 Process discovery

Traditionally, process mining has been focusing on discovery, i.e. deriving information about the original process model, the organizational context, and execution properties from enactment logs. It could be used as a tool to find out how people and/or procedures really work [3].

To give an idea of the capabilities of process mining a simple example is given taken from [6]. In table 1 an event log is shown with, as already mentioned, a case, an activity, an originator and a timestamp. Table 1: Example of an event log [6].

Some results from mining using the event log in table 1 are show in figure 2. Figure 2(a) shows the control-flow structure derived from the event log. The figure shows that the process always starts at activity A and ends with activity D and that if activity B is executed, the also activity C is executed. So after A there is the choice between B and C concurrently (i.e. parallel or in any order) or E, ending with activity D.

Figure 2(b) shows the organizational tasks divided among the people. We can see that activity A is always executed by either John or Sue, activity B is executed by John, Sue, Mike or Carol, this is the same for activity C, D is executed by Peter or Clare and E is executed by Clare. This information can be used to guess/discover the organizational structure. One could guess that there are three roles in this organization unit: X, Y and Z. To be able to execute activity A you should have role X within the organization and John and Sue have this role. In the same way roles Y and Z could be ‘discovered’. Figure 2(c) shows the actual working relationship among individuals. Through the mining process we can derive that even though Carol and Mike can execute the same activities (B and

6

C); Mike is always working with John. In the same manner other actual working relationship can also be derived.

One important note to be made here is that the example shows a small amount of records in the event log. On the basis of such a small amount it is of course not possible to make accurate assumption, but it is to give an idea of the discoveries that can be done through process mining. Real world event logs will contain thousands or more event and those records give more basis for accurate discoveries.

Figure 2: Some mining results from the process perspective (a) and organizational perspective (b) based on the event log in Figure 2 [6].

2.2 Conformance checking

The second type of analysis based on event logs is conformance checking. Unlike process discovery, it is assumed that there is an a-priori model. This model is used to check if reality conforms to the model.

This functionality of process mining searches for inconsistencies between a process model and its corresponding execution log. The fitness between the model and the log is measured (i.e. “Does the observed process comply with the control flow specified by the process model?”) and the appropriateness of the model can be analyzed through checking of the log (i.e. “Does the model describe the observed process in a suitable way?”) [3, 7].

2.3 Extension

The third type of process mining assumes again both a log and a model. However, the model is not checked for correctness, instead it is used as a basis, i.e. the model is extended with a new aspect of perspective. There are different ways to extend a gives process model with additional perspectives based on event logs, e.g. decision mining, performance analysis, and user profiling. Decision mining, also referred to as decision point analysis, aims at the detection of data dependencies that affect the routing of a case. Staring from a process model, one can analyze how data attributes influence the choices made in the process based on past process execution. The process model can also be extended with timing information (e.g. bottleneck analysis) [3].

7

3 EDP-AUDITING

To give you a view about what Electronic Data Process (EDP) auditing is we give you first a definition about EDP auditing:

“It is the independent and impartial appraisal of the reliability, security, effectiveness and efficiency of automated computer systems, the organization of the automation department and the technical/organizational infrastructure of the automated fact processing.”[8]

When we look at the six EDP auditing independent and impartial appraisals we see that these are the main factors for business processes for an EDP auditor. These are the key factors where an EDP auditor is checking a system described in the literature. At the Symposium were we presented our research a lecturer from Ernst & Young, an EDP auditor, had given his lecture about EDP auditing & Innovation. He told us that before he become an EDP auditor he had learned about these six key factors were the basic of EDP auditing was relied on but in the reality it is more than these factors.

The first appraisal, Reliability, gives an EDP auditor the view about in which the business processes are reliable for the automated system. Nowadays the automated systems have more transactions and more processes than before. This gives an EDP auditor more processes to examine at the Reliability of the system.

The second appraisal, Security, gives an EDP auditor the view about in which way the security has been established. His methodology focuses on the analysis of the structure and performance of control processes. Representative transactions are examined by the auditor to assure that these processes are functioning consistently and correctly. An EDP auditor has also different kind of checking tools to checks leaks in the automated system. The goal of the EDP auditor is not to check fraud because the perpetration of a fraud typically manipulates the purpose and content of specific transactions, rather than the process itself. For an EDP auditor this kind of fraud is not to see, because the manipulated transaction is fully blend into normal (legitimate) transaction flow and through the administrative process is being compromised [9].

The third appraisal, Effectiveness, is about in which way an automated system is effective. How is the processes of the system effective regulated. In this way an EDP auditor views the system and gives recommendation in which way the system is effective. To do that an EDP auditor van use scripts or doing it by hand. Nowadays with the rise of the emerging information technologies the use of audit computer-assisted techniques are more effectively used because of the new generation system are using more data mining, object-oriented architecture and intelligent agents processes in the automated system.[10]

The fourth appraisal, Efficiency, is about in which way the processes are optimally regulated. In the efficiency method the EDP auditor looks if resources are optimally used in the automated system. Here the EDP-auditor can recommend whether the resources must be downgraded or be extended to give a better business performance. Nowadays EDP auditors have many specific tools to use like Cobit, but the use of these IT tools is less. Auditors are doing mostly their recommendation by hand and with their use of knowledge.

The fifth appraisal, the organization of the automation department, is the fact in which way for example the segregation of duties is regulated for the employee to check or to place orders for in the system. The EDP auditor looks on this fact how does and how many employees are used for the resources of any business process in the system. He is using his experience and his knowledge to recommend if there are more employees necessary or that it is better to re-engineer your system.

8

The technical/organizational infrastructure of the automated fact processing is about in which way the automated processes are regulated at the company and how this is reliable for the goals of the company. The EDP auditor reviews this and recommends the business processes to be optimal and that the technical infrastructure is regulated in the way that gives the company an efficient and effective way of business performance.

3.1 Work field EDP-auditor

EDP auditor as his main function is to assure that management exercises effective control over the way in which the organizations assets are used and that these factors for business processes is related so that use are current and accurate. Its work is focused on the reasonableness and consistency of the processing methods used and the accuracy, completeness and currency of the data itself, this is called the fairness issue [9].

Also where it is focuses on is the custody and use of organization assets in general. In these both instances significant attention is paid by the EDP auditor to the means used to detect and correct errors. If we look at the business processes at the work field for an EDP auditor we see that much of these factors can be automated to give a better and accurate decision support for an EDP auditor. Since the more using of complex systems and systems that are hand shaped for a company an EDP auditor needs more IT tools to give a better recommendation and spit through the resources for its decision support.

4 PROCESS MINING VS. EDP AUDITING

“Process mining techniques allow for the analysis of business processes based on event logs.”

“EDP-auditing is the independent and impartial appraisal of the reliability, security, effectiveness and efficiency of automated computer systems, the organization of the automation department and the technical/organizational infrastructure of the automated fact processing.”

When analyzing Process mining it is obvious that it is a technique intended to review the business process. The focus is on the business process and its optimization.

EDP-auditing on the other hand is focused on the review of electronic data processing equipment used to support business operations. There is an indirect link with the business processes here. The main objective of EDP-auditing is not optimization of the business processes, but optimization of the IT supporting those business processes. So when having a look at the main goal of EDP-auditing process mining cannot be the primary gear for the EDP-auditor to work with.

Even though process mining tools will not be the primary gear for EDP-auditors, Process mining techniques can still be one of the tools in the toolbox of an EDP-auditor. Through the use of the audit trails (event logs) the EDP-auditor can test whether the information system shows anomalous behavior. And this feature can help the EDP-auditor in, for example, checking for security breaches, checking for effective and efficient data flow through the information system, etc. Appendix A gives an example of how process mining can be used to detect anomalous behavior.

5 CONCLUSION

The main focus of this paper is on the following research question:

9

‘Are there EDP auditing applications for BPM process mining?’

To answer this question we first have taken a look at the new technique of process mining, at the possible analyses that can be done with this technique and at an example to give an idea of what some of the capabilities are. Another important aspect is the field of EDP-auditing. After having a look at the theory of EDP-auditing, we took a small look into the work field of the EDP-auditor. Combining these two and finding similarities was the next step in search of an answer for our main research question. After these step the one thing left to do is answering the main research question.

Are there EDP-auditing applications for BPM process mining? No, not yet. BPM process mining is a young field in which a lot of development is possible. Because of its relatively young status it does not have applications in certain fields. The (possible) use of process mining in the field of EDP-auditing will not be as primary gear, but more as one of the many tools in an EDP-auditing toolbox. The main reason for this is that the goal of both techniques is different, but process mining tools can help the EDP-auditor in certain parts of his job. An example is the use of process mining to detect security breaches in an information system.

6 FURTHER RESEARCH

Process mining tools at the moment are not interesting for EDP-auditor, because the functionalities for the EDP-auditor at the moment are limited and the functionalities offered by the process mining tools are already available in other software tools [11].

Besides the development of process mining the development of EDP-auditing is also interesting to watch. If the EDP-auditor will become more a consultant, analyzing business processes plus its supporting tools and giving feedback on it functioning, then process mining might be more valuable to him. At this moment the EDP-auditor has the role of controller and thus is process mining not the main tool for the EDP-auditor, because of a different focus by both.

Interesting question on this behalf:

Is the IT-auditor an accountant with some IT knowledge or is the IT-auditor an IT-consultant with audit knowledge? And where will EDP-auditing develop into?

10

References [1] http://en.wikipedia.org/wiki/History_of_information_technology_auditing [2] http://ga1717.tm.tue.nl/wiki/ [3] Wil M. P. van der Aalst, Trends in Business Process Analysis: From Verification to

Process Mining. [4] A. Rozinat, R.S. Mans, M. Song and W.M.P. van der Aalst; Discovering Simulation

Models, pages 1-12 [5] Wil van der Aalst, Process Mining and Monitoring Processes and Services: Workshop Report,

The Role of Business Processes in Service Oriented Architectures, pages 1-7. [6] W.M.P. van der Aalst and A.K.A. de Medeiros(2005), Process Mining and Security: Detecting

Anomalous Process Executions and Checking Process Conformance, Electronic Notes in Theoretical Computer Science 121, pages 3-21

[7] A. Rozinat and W.M.P. van der Aalst(2005), Conformance Checking of Processes Based on Monitoring Real Behavior, Group of Information Systems, pages 1-44

[8] http://nl.wikipedia.org/wiki/EDP-Auditing [9] B. Menkus, The EDP Auditor’s Role in Computer Security (1985), Computer & Security 4, North-

Holland, 135-138 [10] Deron Liang, Fengyi Lin, Soushan Wu (2001), Electronically auditing EDP systems with the

support of emerging information technologies, International Journal of Accounting Information Systems, pages 130–147

[11] IS auditing guideline, Use of computer assisted Audit Techniques(CAATs)Document G3, Information Systems Audit and Control Association, pages 1-4

[12] W.M.P. van der Aalst and A.K.A. de Medeiros(2005), Process Mining and Security: Detecting Anomalous Process Executions and Checking Process Conformance, Electronic Notes in Theoretical Computer Science 121, pages 3-21

11

APPENDIX A

Reference: W.M.P. van der Aalst and A.K.A. de Medeiros(2005), Process Mining and Security: Detecting Anomalous Process Executions and Checking Process Conformance, Electronic Notes in Theoretical Computer Science 121, pages 15-17

Imagine a website that is used to sell products. Assume every user in this website has a shopping basket that can be edited at any time. If the shopping basket contains products when the user leaves the website, the user basket’s status is saved and is retrieved when the user enters the website again. Possible user actions are described by the WF-net shown in figure 3. Now, assume we do not know the net in figure 3, but we do have a complete log of acceptable audit trails. For instance, let this audit log be WOK = {“Enter, Select Product, Add to Basket, Cancel Order”, “Enter, Select Product, Remove from Basket, Cancel ”, “Enter, Select Product, Add to Basket, Continue Shopping, Select Product, Remove from Basket, Continue Shopping, Select Product, Add to Basket, Proceed to Checkout, Fill in Delivery Info, Fill in Payment Info, Provide Password, Process Order, Finish Checkout”, “Enter, Select Product, Remove from Basket, Proceed to Checkout, Fill in Payment Info, Fill in Delivery Info, Provide Password, Process Order, Finish Checkout”}. Given WOK as input, the á-algorithm discovers the net shown in Figure 3.

Once the net is discovered, the conformance of every new audit trail can be verified by playing the “token game”. Note that anomalous audit trails do not correspond to possible firing sequences in the “token game” for the discovered net. Furthermore, the “token game” detects the point in which the audit trail diverges from the normal behavior and allows also for the real time verification of trails. For example, let us verify the new audit log WNOK = {“Enter, Select Product, Remove from Basket, Proceed to Checkout, Fill in Delivery Info, Fill in Payment Info, Provide Password, Process Order, Finish Checkout”, “Enter, Select Product, Remove from Basket, Proceed to Checkout, Fill in Payment Info, Fill in Delivery Info, Process Order, Finish Checkout”} by playing every trace in WOK in the net in Figure 4. The first audit trail in WNOK is an acceptable one. Note that this trail is not in WOK, but it can be generated by the discovered net. The second trail is an anomalous one because it does not contain the task Provide Password. By playing the “token game”, we see that two tokens get stuck in the input places of Provide Password. In other words, the “token game” explicitly shows the point where the anomalous behavior happened. The EMiT tool supports the “token game” and indicates deadlocks and remaining tokens. Note that the á-algorithm correctly discovered the net in Figure 3 without requiring the “training” log WOK to show all possible behavior (the first trace in WNOK is not in WOK), although WOK is complete and the first trace at WNOK fits in Figure 3. However, because the á-algorithm aims at discovering the process perspective, it does not capture constraints that relate to data in the system, like the maximum number of times a loop may iterate. For the example in Figure 3, the loop can be executed an unlimited number of times without violating security issues. Nonetheless, if the loop would correspond to user attempts to log into the system, a maximum number of loop iterations must be set. If this is the case, the discovered WF-net must be explicitly modified to incorporate the required data-related constraints. As a final remark, we would like to point out that the simple idea of playing the “token game” can also be used without applying the á-algorithm, i.e., by explicitly modeling the process. However, given the evolving nature of systems and processes, the á-algorithm is a useful tool to keep the “security process” up-to-date. For example, if an audit trail “does not fit” but does not correspond to a violation, then it can be added to the event log used by the á-algorithm. Audit trails that seemed OK, but turned out to be potential security breaches can be removed from the log. By applying the á-algorithm to

12

the modified event log, a new and updated “security process” can be obtained without any modeling efforts.

Figure 3: Example of a process description to buy products at a website

13

INNOVATION & ICT

VRiSBI International Research Project Ireland 2007

Study Association VRiSBI Kamer H11-02

Postbus 1738

3000 DR ROTTERDAM

Email: info@vrisbi.nl

Internet: www.vrisbi.nl

Tel: +31-10-408 8846

Emiel Caron Assistant Professor

Room H10-19

P.O.Box 1738

3000 DR Rotterdam

The Netherlands

Email: caron@few.eur.nl

Tel. +31-10-4081342

Fax. +31-10-408 9162

VRiSBI is the study association for the study Economics & Informatics at the Erasmus University Rotterdam. We have over 350 members and there are around 100 students currently in their final year of the bachelor or master program.

One of our most important tasks is to connect students of Economics & Informatics with companies to give them an inside look how it is in the field. We try to do this by regularly organizing different kinds of activities in association with interested companies.

The development and the pleasure of learning for the student is important to us. We do this by organizing all kinds of activities like company visits, study trips, symposia, etc. etc.

This report in front of you is part of the VRiSBI International Research Project Ireland 2007. The CD-Rom contains all the reports and it also contains the presentations from the symposium ‘Innovation & ICT’.

ISBN of the complete report: 978-90-812660-1-7

VRiSBI International Research Project

“Innovation and ICT”

Comparing Ireland with The Netherlands

Please visit http://studiereis2007.vrisbi.nl for the complete paper of this presentation.

Other papers and presentations are also available.