Development of an expert system for reducing medical errors

International Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013

DOI : 10.5121/ijsea.2013.4603 29

DEVELOPMENT OF AN EXPERT SYSTEM FORREDUCINGMEDICAL ERRORS

Mohamed Ramadan and Khalid Al-Saleh

Industrial Engineering Department, College of Engineering, King Saud University

ABSTRACT

Recent advances in patient safety have been hampered by the hard dealing with the development of auniform classification of patient safety concepts in a systematic way. Therefore, many believe that medicalexpert systems have great potential to improve health care. A framework for computer-based medicalerrors diagnoses of primary systems’ deficiencies is presented. Results of this research assisted indeveloping the hierarchical structure of the medical errors expert system which was written and compliedin CLIPS. It has 225 rules, 52 parameters and 830 conditional paragraphs. The system prompts the userfor response with suggested input formats. The system checks the user input for consistency within thegiven limits. In addition, the system was validated through numerous consultations with the experts in thefield. The benefits that are gained from such types of expert systems are eliminating the fear from dealingwith personal mistake, and providing the up-date information and helps medical staff as a learning tool.

KEYWORDS

Medical errors, expert system, errors classification, patient safety & medical errors diagnoses.

1. INTRODUCTION

In the medical field all kinds of errors are significant, and can have potentially dramatic effects[1, 2]. It was found that the incidence of error included and affected all types of medicalprofessionals at different stages of administration of medical care. This includes physicians,physician's assistants, pharmacists, nurses, and administrative personnel [3]. A complexrelationship exists between medical errors and contributing factors. The same medical error orcircumstance may be perceived as a medical error or a contributing factor, depending on thecontext, circumstance or outcome [4]. A medical error always has a set of contributing factors.Although a medical error can be a contributing factor to the origin or development of anotherincident, some contributing factors can not be incidents in their own right [5,6]. Research showsthat most medical errors are largely preventable [7-9]. Harvard University conducted a study onmedical errors, which led to the recognition that engineers specializing in human factors andhealth systems are needed to improve the existing system [10].

To reduce medical errors, identification and classification of errors must be stated in a clearedstatement [11]. The identification and classification of medical errors in the medical setting is avery complicated process, and this process may be simplified by implementing an effectiveclassification system [12, 13]. In order to reach a consensus on the classification of medicalerrors, it is necessary to develop a generally accepted international medical error classificationsystem [14]. Findings have indicated that errors are likely to affect patients in similar ways incountries with similar healthcare systems [15]. With this taxonomy, the major types of errors canbe categorized and each major type can then be associated with a specific underlying mechanism.


30

This can explain why and even predict when and where an error will occur [16], which in turnwill assist in the generation of intervention strategies for each type of error, and it will assist inthe reduction of medical errors [17-19].

Expert systems are software systems that can be compared to human experts. Their purpose ismostly advisory. Besides, they give explanation and advice to human experts when performingcertain tasks. They are intelligent information systems, and are capable to explain and justify theirconclusions [20]. There are several types of problems that can be solved using knowledge-basedsystems [21-26]. Durkin [27] listed different application areas, including business, whichencompasses marketing, management, finance, accounting, medical, etc. for doing control,design, diagnosis, instruction, interpretation, monitoring, planning, prediction, prescription,selection, detection, evaluation, computation, and classification.

In addition to the expert systems that have applications in different areas of medicine, a variety ofmedical expert systems tools are available and can function as intelligent assistants to clinicians,helping in diagnostic processes, laboratory analysis, treatment protocol, and teaching of medicalstudents and residents [28-33]. Expert systems also have certain bad features such as they can notexamine a patient instead of physicians. Finally, expert system that is good for one certain field isoften not good for another one. They may confuse a physician and make him/her commit wrongdecisions practically under time stress.

The primary goal of this research is to develop expert system applying recent medical errorsclassification and adverse events that are encountered in current medical practice to present someadvice that might help to prevent those medical errors.

2. BACKGROUND

Many patient safety-related classifications were developed all over the world [6]. They were builtusing different techniques and contain different concepts, definitions and terms. Internationally,the agreed definition of patient safety concepts and a uniform approach to classify the conceptshinders comparison of information, learning and system improvement

Currently the World Health Organization’s World league for patient safety has engaged in aproject to develop an International Classification for Patient Safety “ICPS” to develop aclassification system, which converts patient safety information collected from different systemsinto a common format to simplify aggregation, analysis and learning across fields, boundaries andtimes.

Knowledge-based expert systems employ human knowledge to solve problems that normallywould require human expertise. These expert systems represent the expertise knowledge as dataor rules within the computer. These rules and data can be called upon when needed to solveproblems. Books and manuals have a huge amount of knowledge but a human has to read andinterpret the knowledge for it to be utilized. Most of the expert systems are developed viaspecialized software tools called shells. These shells come equipped with an inference mechanism(backward chaining, forward chaining, or both), and require knowledge to be entered according toa specified format (all of which might lead some to categorize CLIPS ver. 6.3 [34] as a shell).Usually they appear with a number of other features, such as tools for writing hypertext, forbuilding friendly user interfaces, for manipulating lists, strings, and objects, and for connectingwith external programs and databases. These shells populate as languages, although certainly witha limited range of application than most programming languages. For more detailed informationon expert system shells, see the "Expert System Shells at Work" series by Schmuller [35].


31

3. DEVELOPING PREVENTIVE MEDICAL ERRORS EXPERTSYSTEM

Based on the above literature, errors in the medical field can occur in many different ways, withpotentially diverse, wide-ranging and hazardous effects. A review of the varied errors related tofundamental areas such as medication, diagnosis, treatment procedures and clerical procedures interms of their number, etiology and possible ramifications, is a complex domain. Consideration ofthe number of possible alterations of specific elements or actions in a health-related setting thatmight be identified as error(s), connected with the large variety of “system” type errors, drives usto conclude that we are dealing with a tremendous range of possibilities.

A classification is an arrangement of concepts into classes and their subdivisions linked toexpress the semantic relationships between them. For example, ‘contributing factors’ precede andperform a role in the generation of any ‘incident type’ (see, Figure 1). An incident can be areportable circumstance, near miss, or harmful incident (adverse event). A reportablecircumstance is a situation in which there was significant chance for harm, but no incidentoccurred (i.e., a busy intensive care unit remaining grossly understaffed during all shift time, ortaking a defibrillator to an emergency and finding it out does not work although it was notneeded). A near miss is an incident which did not reach the patient (e.g., a unit of blood beingconnected to the wrong patient’s vascular, but the error was discovered before the infusionstarted). A no harm incident is one in which an event reached a patient but no discernible harmresulted (e.g., if the unit of blood was infused, but was compatible). A harmful incident is anincident that results in harm to a patient (e.g., the wrong unit of blood was infused and the patientdied from a haemolytic reaction).

Incidents are classified into a number of several types. An incident type is a class made up ofincidents of a widespread nature, grouped because of shared agreed features and is a “parent”class under which many ideas may be grouped. For example, an incident in which an infusionpump was set up wrongly and delivered a sedative overdose, causing respiratory arrest, would beallocated both ‘medication’ and ‘equipment’ incident types. Incident types include clinicaladministration, clinical procedure, documentation, healthcare-associated infection, medication,blood, blood products, nutrition, oxygen, gas, vapour, medical device, medical equipment,behavior, patient accidents, infrastructure fixtures, resources and organizational management [6].

Expert System shell CLIPS is used to design this system. The system classifies errors based on aset of decision rules. Rule-based programming is one of the most commonly used techniques fordeveloping expert systems. In the programming paradigm, rules are used as heuristics, whichspecify a set of actions to be executed and performed for a given situation. In the event that tworules match a given problem situation, the system will employ a conflict resolution strategy tobest resolve the tie based on the specified decision rules. For example, it could break a tie basedon which rule is more specific, or which rule is shorter or based on “refreshing” that is, rules,which had recently been done after conflict resolution, might not be used again for some time inprejudice of new rules.


32

Figure 1. Part of the Network of Classifications of Medical Errors.

Actions taken to eliminate hazard are actions taken to eliminate and manage any future harm, orprobability of harm, associated with an incident. Such actions can influence incidents,contributing factors, and can be pro-active or reactive. Pro-active actions may be identified bymethods such as failure mode and effects analysis [36, 37] and probabilistic risk analysis [16],whereas reactive actions are taken in answering to insights obtained after incidents (e.g. rootcause analysis). Resilience refers to the degree to which a system constantly prevents, detects,alleviates or improves risks or incidents. Resilience allows an organization to get back to itsoriginal ability to supply main functions as soon as possible after incurring harm (injury) [38].

The hierarchical structure of the expert system has been fully developed. It consists of a mainprogram which is written and complied in CLIPS. It has 225 rules, 52 parameters and 830conditional paragraphs of advice. The main sources of information for the development of theknowledge base were taken from different recourses [6, 12, 39-48].

The medical error knowledge domain is based on a variety of rules and utilizes CLIPS shell fromNASA [34]. The knowledge base contains all information that the shell expert system can displaya given material, as well as the information needed to construct and decide the displayed advice.All notifications were collected through consultations with medication staff from Tabouk area, anarea in northern Saudi Arabia, as well as from Kopec et al. [49], Miller [50], and Halbach andSullivan [51]. The data-base sections were divided into control, explanation, and advice logicrules. The purpose of control logic rules is to control where the consultation is and where will itgo next. In addition, it assigns the values of user answers to the data base variables. An exampleof such rules is:

Rule # 71IF incident type = Healthcare Associated InfectionAND


33

Healthcare Associated Infection = Type/Site of infectionANDType/Site of infection = BloodstreamThen1. Compliance with basic prevention measures.2. Active hand hygiene.3. Robust decontamination rather than relying on a single approach.

The explanation logic rules are designed messages at any time the users press how or why keys,or help when they are confused to understand the given question. The following example is asample of the explanation rules:

“Human error in medical practice can be reduced if a culture of awareness emphasizing whenand how errors are prone to occur, in part by implementing Expert System models, is affected.The general nature of human error(s) in complex systems is reviewed and focused on issuesraised by Institute of Medicine, 19991. From this background categorized error(s) in medicalpractice, including medication, procedures, diagnosis, clerical error(s), and others are classifiedthe World Health Organization, 20092. The primary goal of this developed Expert System is toidentify and classify medical errors that are encountered in current medical practice. In addition,we would like to make recommendations for the design of a new type of “preventative system”that will introduce a new “gold” standard for medical systems. This includes physicians,physician's assistants, pharmacists, nurses, administrative personnel, and patient”.

The advice rules express the conclusion that has been reached through the consultation executionsuch as:

Recommendations of clinical process-procedure problems

1. Making patient safety a priority need and theresponsibility of all staff members.

2. Improving the quality of communication among health-careworkers and between patients,

3. Health-care workers can help prevent this error.

The system prompts the user for response with suggested input formats. The system checks theuser input for consistency within the given limits. In addition, the system provides a briefexplanation when the user is confused about particular prompt. Subsequently, the system wasvalidated through numerous consultations with the experts in the field.

4. RESULTS

Consultation Session

Figure 2 shows the information presented as inputs and outputs that appear to the user during aconsultation session with the developed expert system. First, the Medical Errors Clips softwareshould be run using the CLIPS software version 6.3 provided from NASA [34]. Then, the userclicks on the “execution” command and “run” command in order to run the program.

Now the Expert System is ready for the consultation. If a medication incident exist and counter-measure for such type of incident are needed. Then the user needs to type “medication” and press


34

Enter key. Figure 3 will then appear; and one of the causes of medication incident may be typedsuch as “wrong drug”, for example. Enter key must then be pressed again. A window showing theexpert recommendations will be displayed as shown in Figure 4. At the end of this session, theexpert system may be run again by just clicking on the “execution” command and “run”command in order to run the program again.

5. DISCUSSIONS AND CONCLUSIONS

Results of this research assisted in developing the hierarchical structure of an expert systemwhich consists of main program written and complied in CLIPS. The system prompts the user forresponse with suggested input formats. The system checks the user input for consistency withinthe given limits. In addition, the system provides a brief explanation if the user is confused aboutparticular prompt. The period of knowledge acquisition lasted about 9 months; development ofdata bases required 5 months. Subsequently, the proposed medical errors expert system wasevaluated and validated through numerous consultations with the experts in the field testing.

This study achieved its goal by developing expert system software to prevent medical errors. Thisstudy represents a continuous learning and improvement cycle emphasizing risk identification,prevention, detection, and reduction; incident recovery and system resilience. The developedexpert system provides actions that should be taken to reduce risk. These actions concentrate onsteps taken to prevent the reoccurrence of the same or similar patient safety incident and onimproving system resilience. Actions taken to reduce hazard are those actions taken to minimizeand control the hazard, or the probability of harm associated with an incident. These actions maybe directed toward the patient (e.g., decision support), toward staff (e.g., training, availability ofpolicies/procedures), toward the organization (e.g., improved leadership and guidance, proactivehazard assessment), and toward therapeutic agents and equipment (e.g., regular audits).


35

Figure 2. Appearance of starting expert system screen.

Figure 3. Input facts of a case study screen.

Figure 4. Output fact of a case study screen.

The following items are desiderata for responsible and appropriate medical errors classificationexpert systems: mechanisms of action; documentation of nomenclature; required training andskills for safe usage; mechanisms for obtaining help; intended uses and indications;contraindications and limitations to use; software version numbers and system function history;documentation of processes for building and maintenance of the system’s knowledge base;


36

expected duration of validity, existence of pre-programmed expirations, and suggested frequencyof updates; record of previous evaluations; conformance to standards; and problem and errorreporting.

The medical errors occurred by medical practitioners has received considerable attention in recentyears. Yet there is a fundamental conflict amongst experts as to whether the number of deathscaused by these errors has been either under or over-estimated. Regardless, the number of deathscause by such errors is frightening. Therefore, the nature of medical errors was analyzed,including errors in medications, treatment procedures, diagnosis, and clerical functions. Thisnaturally led to our trial to extend the classification of medical errors. In addition we havediscussed the possible past and future roles of computer software applications in reducing therates of such errors. Developments and changes in software applications today offer unbelievableopportunities for “error-prevention and error-reduction.” Hence we are optimistic that greatprogress can and will be made in this field.

Acknowledgements

This research was funded by King Abdel-Aziz for Science and Technology, Saudi Arabia, GrantAT# 26-39.

REFERENCES

[1] Canadian Institute for Health Information. (2004) Report for Healthcare in Canada, Ottawa: Canada.[2] Agency for Healthcare Research and Quality (AHRQ) (2008). The Fifth Annual Health Grades

Patient Safety in American Hospitals Study. Obtained through the Internet:http://www.ahrq.gov/clinic/ptsafety/pdf/ptsafety.pdf, [accessed 7/10/2009].

[3] Baker, G., Norton, P., Flintoft, V., Blais, R., Brown, A. & Cox, J. (2004) “The Canadian AdverseEvents Study: the incidence of adverse events among hospitals in Canada” Canadian MedicalAssociation Journal, Vol.170, pp.1678-1686.

[4] Goodman, F. (2004) “A fragmented patient safety concept: The structure and culture of safetymanagement in health care” Nursing Economics, Vol. 22, No.1, pp.44-46.

[5] Fogarty, G. & McKeon, C. (2006) “Patient safety during medication administration: The influence oforganizational and individual variables on unsafe work practices and medication errors” Ergonomics,Vol. 49, No. (5-6), pp.444-456.

[6] World Health Organization, World Alliance for Patient Safety (2009). The InternationalClassification for Patient Safety, Final Technical Report, Geneva: Switzerland.

[7] Meadows, M. (2003) “Strategies to reduce medical errors”, FDA Consumer, Vol.37, No.3, pp. 20-27.[8] Carayon, P., Gurses, A. & Hundt, S. (2005) Performance obstacles and facilitators of healthcare

providers, In: Korunka, C. and Hoffman, P. (ed.), Change and quality in human service work, Vol. 4.Munchen, Germany: Hampp Publishers.

[9] Rideout, K. (2006) Adverse medical event reduction technologies: identification of primary riskfactors of adverse medical events using Artificial Neural Networks. Ph.D. thesis, Carleton University,Canada.

[10] Gavande, R. (1997). Medical Errors- How can they be reduced. Obtained through the Internet:http://www.engr.wisc.edu/wiscengr/Apr03/medical_errors.shtml, [accessed 9/6/2007].

[11] Morimoto T., Gandhi, T., Seger, A., Hsieh, T. & Bates, D. (2004) ‘Adverse drug events andmedication errors: detection and classification methods’ Quality & Safety in Health Care, Vol. 13, pp.306–314.

[12] Chang, A., Schyve, P., Croteau, R., O’Leary, D. & Loeb, J. (2005) “The JCAHO patient safety eventtaxonomy: a standardized terminology and classification schema for near misses and adverse events”International Journal of Quality in Health Care, Vol. 17, pp.95-105.

http://www.ahrq.gov/clinic/ptsafety/pdf/ptsafety.pdf

http://www.engr.wisc.edu/wiscengr/Apr03/medical_errors.shtml


37

[13] Unruh, K. & Pratt, W. (2006) ‘Patients as actors: The patient’s role in detecting, preventing, andrecovering from medical errors’ International Journal of Medical Informatics, Vol. 76, No. 1,pp.S236–S244.

[14] World Health Organization (2006). International Statistical Classification of Diseases and RelatedHealth Problems. 10th Revision. Version for 2006 (ICD-10). Obtained through the Internet:http://www.who.int/classifications/icd/en/index.html, [accessed 10/6/2008].

[15] Espin, S. (2006) Error and reporting in surgery: exploring team and patient perceptions. Ph.D thesis,Department of Institute of Medical Science, University of Toronto, Canada.

[16] Institute for Safe Medication Practices Canada (2006). Canadian Medication Incident Reporting andPrevention System. Definitions of Terms. Obtained through the Internet:

[17] Crane, J. & Crane, F. (2006), “Preventing Medication Errors in Hospitals through a SystemsApproach and Technological Innovation: A Prescription for 2010” Hospital Topics, Vol. 84, No.4,pp.3-8.

[18] Elfering, A., Semmer, N. & Grebner, S. (2006) “Work stress and patient safety: Observer-rated workstressors as predictors of characteristics of safety-related events reported by young nurses”Ergonomics, Vol. 49, No.(5-6), pp.457-469.

[19] Hogan, C. A. (2006) Pediatric Patient Safety: Factors pediatric nurses identify as contributing tomedication administration errors. Ph.D. thesis, Nursing School, Loyola University, Chicago, Illinois,U.S.A.

[20] Hadi, W. (2013) “EMCAR: Expert Multi Class Based on Association Rule”, International Journal ofModern Education and Computer Science, Vol.3, pp. 33-41.

[21] Kitporntheranunt, M., Wiriyasuttiwong, W., Development of a medical expert system for thediagnosis of ectopic pregnancy. Journal of the Medical Association of Thailand, 2010, 93 Suppl 2, pp.S43-49.

[22] Kumar, D.S., Sathyadevi, G., & Sivanesh, S. (2011) “Decision Support System for Medical DiagnosisUsing Data Mining”, International Journal of Computer Science Issues, Vol. 8, Issue 3, No. 1, pp.147-153.

[23] Chang, R., (2011) “Interactive E-diagnosis: An Efficient Scheme for Medical Diagnosis SupportSystem”, Journal of Networks, Vol. 6, No. 5, 2011, pp. 713-720.

[24] Santosh, P. & Patra, K. (2012) “Automatic Diagnosis of Diabetes by Expert System” InternationalJournal of Computer Science Issues, Vol. 9, Issue 2, pp. 299-304.

[25] Gonz´alez, A., Ni˜no, J. & Mayer, M.A. (2012) “Hernandez, G.A., Wilkinson, M.D., Analysis of aMultilevel Diagnosis Decision Support System and Its Implications: A Case Study”, Computationaland Mathematical Methods in Medicine, Volume 2012, Article: ID 367345, pp. 1-9.

[26] Chi Fai David Lam, Kwong Sak Leung, Pheng Ann Heng, Chi Eung Danforn Lim & Felix Wu ShunWong, (2012) “Chinese Acupuncture Expert System (CAES)—A Useful Tool to Practice and LearnMedical Acupuncture” Journal of Medical Systems, Vol. 36, pp.1883–1890.

[27] Durkin, J. (1993) Expert Systems: Catalog of Applications, Akron: OH.[28] Durinck, J., Coiera, E. & Baud, R. (1994) “The role of knowledge based systems in clinical practice”,

In Barahona, P. and Christensen, J.(ed.), Knowledge and Decisions in Health Telematics - The NextDecade, (pp. 199-203) Amsterdam: IOS Press.

[29] Benner, P., Sheets, V., Uris, P., Malloch, K., Schwed & Jamison, D. (2002) “Individual, practice, andsystem causes of errors in nursing” Journal of Nursing Administration, Vol. 32, No. 10, pp.509-523.[

[30] Santhiyakumari, N., Rajendran, P. & Madheswaran, M. (2011) “Medical Decision-Making System ofUltrasound Carotid Artery Intima–Media Thickness Using Neural Networks” Journal of DigitalImaging, Vol.24, pp.1112–1125.

[31] Tomar, P.P., Singh, R. & Saxena, P.K. (2012) “A Medical Multimedia based Clinical DecisionSupport System for Operational Chronic Lung Diseases Diagnosis and Training”, InternationalJournal of Computer Applications, Volume 49, No.8, pp. 1-12.

[32] Munirah, M., Rozlini, S.F., Chew, (2012) Design and Rules Development of Online Children SkinDiseases Diagnosis System, International Conference on Information and Knowledge Management(ICIKM 2012) IPCSIT vol.45, pp.272-276.

[33] Das, S., Chowdhury, S.R. & Saha, H. (2012) “Accuracy Enhancement in a Fuzzy Expert DecisionMaking System Through Appropriate Determination of Membership Functions and Its Application ina Medical Diagnostic Decision Making System”, Journal of Medical Systems, Vol. 36, pp.1607–1620

http://www.who.int/classifications/icd/en/index.html


38

[34] CLIPS Software ver. 6.3. (2009). Obtained through the Internet: http://clipsrules.sourceforge.net/ &http://sourceforge.net/projects/clipsrules/files/, [accessed 8/5/2009].

[35] Schmuller PC AI. (1991, 1992). "Expert System Shells at Work" series. Obtained through theInternet: http://www.pcai.com/web/ai_info/expert_systems.html, [accessed 1/12/2000].

[36] National Coordinating Council for Medication Error Reporting and Prevention. (1998) NCC MERPTaxonomy of Medication Errors, Rockville, MD: Office of the Secretariat, United States.

[37] Gawande A., Zinner, M., Studdert, D. & Brennan, T. (2003) “Analysis of errors reported by surgeonsat three teaching hospitals” Surgery, Vol.133, pp.614-21.

[38] Trucco, P. & Cavallin, M. (2006) “A quantitative approach to clinical risk assessment: The CREAmethod” Safety Science, Vol. 44, pp. 491-513.

[39] National Health Service (2007). The National Reporting and Learning System National Patient SafetyAgency. Obtained through the Internet: http://www.npsa.nhs.uk/nrls/reporting/, [accessed 10/6/2008].

[40] Runciman, W., Williamson, J., Deakin, A., Benveniste, K., Bannon, K. and Hibbert, P. (2006) ‘Anintegrated framework for safety, quality and risk management: an information and incidentmanagement system based on a universal patient safety classification’ Quality and Safety in HealthCare, Vol.15, No.1, pp.i82-90.

[41] World Health Organization, Family of International Classifications Overview (2004a). Obtainedthrough the Internet: http://www.who.int/classifications/en/, [accessed 9/6/2006].

[42] World Health Organization, Drug Dictionary (maintained by the Uppsala Monitoring Centre)(2004b). Obtained through the Internet: http://www.who.int/medicines/services/medicines_etools/en/,[accessed 9/6/2006].

[43] World Alliance for Patient Safety (2005). WHO Draft Guidelines for Adverse Event Reporting andLearning Systems. Geneva: World Health Organization (WHO/EIP/SPO/QPS/05.3).

[44] World Health Organization, Alliance for Patient Safety (2007a). Report on the Results of the Web-Based Modified Delphi Survey of the International Classification for Patient Safety. Geneva:Switzerland.

[45] World Health Organization, Alliance for Patient Safety (2007b). Report of the WHO World Alliancefor Patient Safety Meeting with Technical Experts from the South East Asian and Western PacificRegions of the WHO, Tokyo: Japan.

[46] World Health Organization, Alliance for Patient Safety. (2008a). Report of the WHO World Alliancefor Patient Safety Challenge Group Meeting - Validity Testing of the Conceptual Framework for theInternational Classification for Patient Safety, Geneva: Switzerland.

[47] World Health Organization, Alliance for Patient Safety (2008b). Report of the WHO World Alliancefor Patient Safety Meeting with Francophone Technical Experts – Cultural and Linguistic Evaluationof the Conceptual Framework for the International Classification for Patient Safety, Paris: France.

[48] World Health Organization, Alliance for Patient Safety (2008c). Report of the WHO World Alliancefor Patient Safety Meeting with Spanish and Latin American Technical Experts – Cultural andLinguistic Evaluation of the Conceptual Framework for the International Classification for PatientSafety, Madrid: Spain.

[49] Kopec, D., Kabir, H., Reinharth, D., Rothschild, O. & Castiglione, J. (2003) ‘Human Errors inMedical Practice: Systematic Classification and Reduction with Automated Information Systems’Journal of medical system, Vol. 27, No.4, pp.297-313.

[50] Miller, S. (2003) Medical Error Prevention. Health Studies Institute, Inc. Deerfield Beach: Florida,U.S.A.

[51] Halbach, J. & Sullivan, L. (2003) A Curriculum Guide for Teaching Medical Students and FamilyPractice Residents, 3rd edition. Department of Family Medicine New York Medical College Valhalla:New York.

http://clipsrules.sourceforge.net/

http://sourceforge.net/projects/clipsrules/files/

http://www.pcai.com/web/ai_info/expert_systems.html

http://www.npsa.nhs.uk/nrls/reporting/

http://www.who.int/classifications/en/

http://www.who.int/medicines/services/medicines_etools/en/