FROM CONFORMANCE TO PERFORMANCE AUDITING AND

CONTINUOUS IMPROVEMENT OF ISO 9000 QUALITY CERTIFIED

SYSTEMS

Dr Milé Terziovski, PhD* Director and Deputy Chair, European Australian Cooperation Centre, Faculty of

Economics and Commerce, The University of Melbourne, Parkville, Victoria, Australia,

el: 61 3 93447868, Fax: 61 3 8344 3714, Email: milet@unimelb.edu.au

Mr Damien Power Lecturer, Department of Management, Faculty of Business and Economics,

Monash University, P.O. Box 197, Caulfield East VIC 3145 Australia, Tel: 61 3 9031030, Fax: 61 3 99032718, Email: Damien.Power@BusEco.monash.edu.au

Professor Amrik S. Sohal, PhD

Associate Dean research, Faculty of Business and Economics, Monash University, P.O. Box 197, Caulfield East VIC 3145, Australia, Tel: 61 3

99032033, Fax: 61 3 99032718, Email: Amrik.Sohal@BusEco.monash.edu.au

ABSTRACT There has ben a major push by firms to seek certification to the International Quality Standards ISO 9000. This heightened interest is not without problems as many managers have misunderstood the value of seeking ISO 9000 certification. A review of the literature revealed a major gap in research in this area of quality management. Therefore, the aim of this study is to examine the role of the non-financial auditor and the audit process with respect to the existing ISO 9000 Quality standards. Qualitative and quantitative research design was used. Five case studies were developed of certified companies. Two survey questionnaires were designed and administered to 300 auditors registered by JAS-ANZ (Joint Accreditation Service of Australia and New Zealand) and 1500 companies certified by Quality Assurance Services. A response rate of 42% was obtained for the Auditor questionnaire and 28% for the certified organisations. A four quadrant model was developed to categorise companies according to perceived value derived from certification (performance versus conformance) and the maturity level of their continuous improvement culture (ISO 9000 versus continuous improvement). We conclude that conformance auditing has a role in the early stages of quality systems implementation, however effectiveness diminishes as the quality system matures.

KEY WORDS: Performance, Auditing, Continuous improvement.

2

* Corresponding Author

INTRODUCTION

In recent years, there has been a major push by firms to seek certification to the International Quality Standards ISO 9000. These standards have been promoted as the passport required to conduct business in the global marketplace and as a path to Continuous Improvement. Specifically, the ISO 9000 standards have been designed to assist organisations in the implementation and maintenance of quality systems so that an organisation has in place: (i) a quality policy; (ii) standardisation of processes; (iii) system for monitoring of defects and corrective action; and (iv) management reviews of the system. While the ISO 9000 quality certification systems may be a path to TQM, they are only one part of a larger picture. For example, the Australian Quality Award allows 15 per cent of its marks for the quality of products and processes of which ISO 9000 certification is a part. The US Baldrige Award and the European Quality Award give 14 per cent of their points for quality systems. Despite the international consensus, many managers still believe and act as if the process of gaining ISO 9000 certification is the same as becoming a quality organisation. This misconception has raised international concerns, so much so that the European Commission Directorate General for Industry has questioned the effectiveness of ISO 9000 certification as a tool to drive quality [18]. The problem is exacerbated by the lack of rigorous research on the strength of the relationship between ISO 9000 certification and organisational performance. The limited evidence is contradictory and anecdotal [1,5]. The Karpin Report [11] stated that: “There is a strong view internationally that those companies which take their commitment to quality beyond the certification stage into a mode of continuous improvement find that the empowered and flatter organisational model emerges as the most effective means of achieving improved quality and customer satisfaction.” LITERATURE REVIEW The aim of the literature review is to identify and highlight the important variables from the literature, and to document the significant findings from earlier research that will serve as the foundation for the current investigation. The literature review identified several hundred articles concerning ISO 9000 certification in quality management and related journals. Many of these articles were found to be anecdotal and lack statistical and theoretical support. However, several empirically based articles were found. For example, Sun [] investigates ISO 9000 certification and its relationship to the improvement of business performance. The research is based on survey results of 363 companies in Norway who are

3

members of the Norwegian Quality Association (NFK). Findings suggested that ISO 9000 certification was positively related to some aspects of quality performance including reducing product defects and company complaints. However certification did not relate to critical strategic plans regarding quality and human resource development, which are significantly correlated to performance improvement. Sun concludes that a company must integrate ISO 9000 procedures with the human resource and strategic management processes of TQM in order to achieve continuous improvement. Sun raises implications of the research including the need for ISO 9000 to adopt performance criteria associated with TQM in addition to conformance to standard procedures. Such findings raise the need for the auditor to include a role which assists in the development of managers and employees as well as procedures.

Swift et al [] provide a theoretical critique of the practice of the quality audit highlighting it�s parallel to movements in the financial audit to embrace a more value added holistic approach. Swift et al suggest that in order for quality auditors to avoid obsolescence there has been and will continue to be a movement toward closing the �audit expectations gap� which signifies a variance between the auditors performance and the desires of for whom the audit is being carried out. In their analysis Swift et al [] show the importance of the changing role of the auditor from that of a policing activity to one that assists in the creation of continuous improvement. Their analysis identifies the particular concern for those companies which are mature in the certification process who in turn receive diminishing value from a conformance based quality audit. The quality auditor�s role is and should be moving toward an inclusion of human relation principals and organizational culture in order to offer recommendations and assist a company in adding value.

Karapetrovic and Willborn [] in their analysis of the effectiveness of quality audits prescribe that quality assurance of auditing services must be examined and revised in order to achieve continuous improvement objectives. Once again conformance-based audits are scrutinized and new methodology for conducting quality audits is recommended. Karapetrovic and Willborn [] conceptualise a system of interdependent audits which should be established in order to identify non-conformance and areas for improvement which can be addressed at a sequential audit so that progress may be reviewed. Such a systems paves the way for continuous improvement within an organisation. Flexibility, little bureaucracy and adaptation to environmental change are rendered critical in ensuring the effectiveness of the audit system. Manager's Motives for Adopting ISO 9000 Certification A major UK-based study was conducted by the Science and Engineering Policy Studies Unit [16] to identify the reasons why organisations become certified and to ascertain the effectiveness of ISO 9000 certification. The study identified 28 surveys which showed a large variation in the results. There appears to be a relationship between the manager's motives for adopting ISO 9000 certification and the ensuing experience. Those organisations that pursue ISO 9000 certification willingly and positively across a broad spread of objectives are more likely to

4

report improved organisational performance than those organisations which are pursuing ISO 9000 certification because of customer pressure. This was found to be the most common motivating factor for pursuing ISO 9000 certification and yet the least likely to produce a satisfactory outcome. The fundamental problem with obtaining ISO 9000 certification is perception based. Managers perceive the certificate as an end in itself rather than a means to an end. This view is supported by Binney [4] who conducted a qualitative study on the implementation of ISO 9000 quality certification systems. He comments on his findings: " ISO 9000 confirms that a company has an effective quality management system. It does not guarantee that the goods and services the company produces are of quality; that depends on whether the systems serve the interests of customers and are supported by a quality culture. A number of managers commented on companies which in their view supplied poor quality but which nevertheless had been certified for ISO 9000." The Business Value of ISO 9000 Certification Binney's qualitative findings imply that a company may be certified to ISO 9000 and still provide poor quality products and services and not necessarily increase its market share, improve the motivation of its staff or reduce its cost. Batchelor [3] identified similar results in a quantitative study of 647 certified manufacturing and service organisations. He found that 15 per cent of the respondents had derived business value on nine dimensions: market share, new customers, customer satisfaction, procedural efficiency, staff motivation, staff attitudes, error rates, wastage and costs. The benefits attributable to ISO 9000 certification were for procedural efficiency and error rates, and least likely for market share, staff motivation and costs. On the other hand, Terziovski et al [19] based on a large cross-sectional study in the Australian and the New Zrealand manufacturing industry found that ISO 9000 certification is not shown to have a significantly positive effect on organisational performance in the presence or absence of a TQM environment. This supports the view that on average ISO 9000 certification has little or no explanatory power of organisational performance. The majority of the anecdotal articles have been written by third party certifying organisations, consulting firms and practitioners in the field [17,13,8]. These articles argue that the primary business value of ISO 9000 certification lies in the ability of the certificate to open doors to markets which were previously closed. For example, a study commissioned by Lloyd's Register Quality Assurance [12] found that most of the benefits associated with ISO 9000 certification were external. The results show that sixty nine percent of the managers said that the ISO 9000 certification enabled them to improve their business performance by allowing them to bid for tenders from which they were otherwise excluded. The Role of ISO 9000 Certification in Creating Quality Organisations

5

Why do managers misunderstand the role of ISO 9000 certification? One possible explanation is that managers fail to distinguish between conformance and performance specification. ISO 9000 certification is a conformance specification which demonstrates that the organisation�s systems are potentially capable. On the other hand, performance specification requires realistic targets to be set and pursued by employees who are capable and willing to use the ISO 9000 quality certification system. A further explanation is provided by Askey and Dale [2]. They studied the change in attitude and behaviour of managers in organisations that were certified to ISO 9000. They found that when the goal of ISO 9000 certification was achieved, managers tended to revert to their traditional practices. That is, no permanent change in attitude and behaviour had been achieved. Managers were found to revert to 'fire-fighting' rather than planning and engaging their workforce in preventive actions and continuous improvement. They were more concerned about following the rules of the ISO 9000 standard than satisfying customer needs [20,1,5]. Non-Financial Auditing The majority of the literature relating to non-financial auditing has focussed on the auditing of quality systems. The �Quality Audit� is formally defined in ISO 10011-1:1990 - Guidelines for Auditing Quality Systems, as:

“… a systematic and independent examination to determine whether quality activities and results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable for achieving objectives” [9].

Mills [14] considers that the quality audit provides the client with an independent assessment of conformance and the effectiveness of the organisation�s operating systems against predefined standards. He also contrasts the differing focus of internal and external audits. The internal audit evaluates conformance to the organisation�s operating procedures, and the effectiveness of these procedures, while the external audit determines conformance to some predefined standard as established by the certifying body (such as ISO 9000 series). Mills also notes that one of the major problems confronting auditors is that of maintaining consistency in the interpretation of the standards between differing sizes and types of organisations. Ingman [6] takes the view that the quality audit should be used as a means of strengthening the quality system by removing barriers that may impede the continuous improvement process. Rather than a bureaucratic process he sees it as a tool that can be used specifically to remove what he describes as �..bureaucratic encumbrance, the cancer of any quality system�. To facilitate this he believes the use of feedback and involvement of all parties is essential. In summary he states:

“The people part of the auditing process is extremely important. Knowledge of the standard alone is a long way from conducting

6

effective quality audits. People make quality happen. And people make the quality audit effective” [7].

Russell and Regel [15] take this view further by developing a formal process to ensure that the audit acts to facilitate this type of improvement. They state that many clients believe that the audit process is complete when the audit report has been delivered, and that this signals the end of the involvement of the auditor in the process. In fact they believe that this constitutes at most three-fourths of the process, with the most important phase of implementation and verification of corrective actions constituting the final, and in their view most important, stage. They focus on this stage and recommend the use of an �Audit Function Improvement Process� to provide techniques for improving the corrective action process. They believe that the primary reason for breakdown in the audit process stems from a lack of involvement of the auditor in the implementation of corrective actions, and that:

“The interaction between auditor, auditee, management, and other stakeholders is beneficial in verifying their understanding of the facts in the early stages of the correction process [15].

Summary The review of the literature indicates that ISO certification can deliver business value, but should be seen to be part of an overall continuous improvement process rather than an end in itself. The literature also points to the appropriate role of the auditor in facilitating this process, yet there is a lack of evidence as to the real impact non-financial auditors have. Motivation for certification and the quality culture of the organisation are identified as important factors, yet there is little empirical evidence linking them to business outcomes in the context of certification. Identifying the linkages between auditing style and conduct, quality culture, motivation for certification and business value have therefore been identified as appropriate for the focus of this paper. HYPOTHESES In order to develop a framework for the investigation the following research questions were developed:

• In the context of certification what is the relationship between quality culture, motivation for certification auditing style and business performance?

• Is auditing style a determinant of the quality culture of certified

organisations?

• Is quality culture a determinant of the business value of certification?

• Does the motivation to seek certification to ISO 9000 standards have an impact on its effectiveness?

7

Considering the literature findings this study will examine the working hypotheses that:

• “Organisations that pursue ISO 9000 certification willingly and positively with a strong quality culture are more likely to report organisational benefits than those organisations which are pursuing ISO 9000 certification reactively with a weak positive culture”

• ”Non-financial auditors can contribute to the development of quality

cultures in certified organisations.” Based on the working hypotheses three research hypothesis will be tested. Research Hypothesis H1: There is a significant association between the quality management culture of an organisation and the benefits derived from certification. H2: There is a significant association between the style adopted by the auditor and the quality management culture of the organisation. H3: There is a significant association between the motivation for certification and the benefits derived from certification. METHODOLOGY Data Gathering The methodology used to test these hypotheses employed a combination of survey research and case studies. Two survey instruments were designed, one for non-financial auditors and the other for companies who are clients of non-financial auditors. The review of the literature provided the theoretical basis for the development of the major constructs contained in the questionnaires. These constructs were further tested through a survey trial and by a process of academic review. This trialing procedure enabled the survey instruments to be further refined and statistical testing to be performed to establish confidence in the validity and reliability of these constructs. The major areas covered by these instruments included:

• The appropriate role, style and skills required for effective non-financial auditing.

• The purpose of the audit and what constitutes a successful audit. • Professional knowledge and experience required by non-financial

auditors and recent training undertaken. • Value derived from ISO certification. • Quality culture of the organisation. • Client practice during auditing. • Reasons for seeking certification.

8

• Auditor and client organisation demographics. 300 non-financial auditors were surveyed with 126 responding, (response rate of 42%), while 1500 companies were surveyed with 400 responding (response rate of 27%). The results from the survey were analysed using the SPSS (Statistical Package for the Social Sciences) program. In analysing this data a combination of techniques have been used. The results reported here have been derived from the use of factor analysis and multiple regression. The measure used for comparison is �adjusted R squared�, a measure of association that is rigorous in enabling comparison between samples of differing sizes containing different numbers of independent variables. The case studies were conducted using a protocol developed from the initial survey results. This protocol covered the following major issues:

• Organisational background. • Certification history. • Current audit practice.

o The role, style etc. of the auditor. o Benefits from certification. o Limitations of certification. o Future trends in non-financial auditing and certification. o Issues particular to that organisation of relevance (eg cultural etc.).

The cases were conducted in five organisations that had participated in the survey, and data was gathered using a combination of interviews, observation and documentation. Framework for Data Analysis In order to provide a background against which to analyse the data gathered the following four quadrant model was developed capturing the value derived from certification (performance) and the quality culture of the organisation. INSERT FIGURE 1 This model was developed for comparison of the hypotheses across groups of certified organisations based on their progress along the �quality journey�. The organisations in the top and bottom halves of the quadrant are therefore separated on the basis of their perception of the contribution of ISO to performance, while the separation between the left and right side is on the basis of the maturity of the organisation�s quality culture. An explanation of the four strategies typical of organisations within this framework follows.

9

THEORETICAL FRAMEWORK There are four strategy types which organisations typically fit into when gaining certification to ISO 9000. These strategy types are briefly explained below: Strategy Type 1- Proactive ISO 9000 Strategy An organisation commits to a Type 1 strategy when it proactively seeks ISO 9000 certification but still has a developing quality culture. This strategy is likely to produce short term organisational gains due to the novelty and enthusiasm of becoming certified. Brown & van der Wiele [5] conclude that making people think about quality or improving systems prepares the way for further progress in improving quality. Strategy Type 2 - Reactive Pursuit of ISO 9000 An organisation usually commits to a Type 2 strategy when it gains certification to ISO 9000 standards as a result of customer pressure or government purchasing policy, and has a weak quality culture. This group do not typically see ISO as contributing significantly to business performance. Strategy Type 3 - Non- ISO 9000 Continuous Improvement Focus An organisation often commits to a Type 3 strategy when it embraces continuous improvement before attempting ISO 9000 certification. These organisations typically believe that creating a Continuous Improvement culture is a necessary precursor to seeking ISO 9000 certification, and that ISO 9000 is not a major contributor to organisational performance. Strategy Type 4 - Proactive Pursuit of Continuous Improvement and ISO 9000 Certification An organisation commits to a Type 4 strategy when it adopts a Continuous Improvement culture first and then ISO 9000 certification. These organisations are usually characterised by a combination of both a strong quality culture focused on Continuous Improvement and ISO 9000 certification creating a best practice organisation.

QUANTITATIVE AND QUALITATIVE RESULTS Quantitative Results H1: There is a significant association between the quality management culture of an organisation and the benefits derived from certification. The initial data analysis revealed only two cases where a significant association of moderate strength was found between the independent variable �Quality Culture� and the benefits derived from certification. These are contained in Table 1 below:

10

INSERT TABLE 1 The two cells highlighted in bold type in Table 1 indicate a moderately strong relationship at a high degree of significance. Across all companies there was one case of a moderate association found between quality culture and contribution of ISO to improved business performance, but no other significant association between the other variables. Across the four groups (strategies) of the quadrant, only the Pro-active group reported an association of any significance and strength, that being between Management Responsibility and the contribution of ISO to waste reduction. This hypothesis was further tested by breaking the groups up according to the number of years they have been certified, and the results are contained in Table 2 below: INSERT TABLE 2 Table 2 shows there to be strong and significant relationships between all four quality culture and ISO related performance outcomes in the organisations certified for less than five years in the Pro-active group. In the companies within this group certified for more than 5 years there was one instance of a strong and significant relationship. In the integrated group there was only one instance of a moderate association, that being in the group of organisations certified from 2-5 years, while there were no significant relationships recorded in either the Reactive or the Pro-active groups. These results indicate that the relationship between quality culture and ISO related performance is stronger in the top half of the quadrant than the lower half. It is particularly interesting to note the relationship between the length of time certified and the impact of quality culture on ISO related performance. It can be hypothesised that as organisations become more mature in the operation of their quality systems, the relationship between ISO certification (as it is now) and the quality culture of the organisation changes and becomes more �seamless�. It is also evident that the relationship is dynamic, possibly indicating the need for different approaches to non-financial auditing and certification for different types of organisations. H2: There is a significant association between the role and style adopted by the auditor and the quality management culture of the organisation. To test this hypothesis there were two separate auditing styles identified through the survey. Auditing for continuous improvement was characterised by an approach that focused on gathering data to assist clients in improving their quality systems,

11

helping clients improve people, products, processes and services and being flexible in order to best meet the need of the client. Auditing for compliance on the other hand captured issues such as solely verifying performance against the standard, a focus on inspection and surveillance and an inflexible, detached style. The initial data analysis under this hypothesis revealed only three cases where a significant association of moderate strength was found between the independent variables relating to auditing style and quality culture. These are contained in Table 3 below: INSERT TABLE 3 The three cases where these associations were noted are all in the bottom half of the quadrant, among the Reactive and Continuous Improvement groups. This hypothesis was further tested by breaking the groups up according to the number of years they have been certified, and the results are contained in Table 4 below: INSERT TABLE 4 Four strong and significant associations were found through this analysis, with three of the four occurring in the Reactive and Continuous Improvement groups (the bottom half of the quadrant), and among companies certified for both less than and greater than 5 years. The Integrated quality Strategy group showed no instances of any association between auditing style and quality culture irrespective of number of years certified. The most interesting aspect of these results is the lack of any association in the Integrated grouping. This leads to the possible conclusion that organisations with mature quality cultures see the style of the auditor, and by extension the audit process, as of little impact on their quality culture. Organisations with immature or developing quality cultures (Reactive or Proactive groups), and those with strong cultures but reporting little benefit from certification (Continuous Improvement group) appear to have a much stronger interaction with the auditor. As organisations move into the Integrated grouping, the seamless nature of the relationships reported in the analysis of Hypothesis 1 appear to extend to the style and influence of the auditor. H3: There is a significant association between the motivation for certification and the benefits derived from certification. In order to test this hypothesis three types of motivational strategies were identified. A Continuous Improvement strategy included motivators such as ISO being part of a TQM based strategy, the desire to provide better products and processes and the

12

recognition of the need to provide better products and processes. The second, characterised as reaction to environmental factors, included changes in legislation and changes in industry policies and trends. The third, reaction to pressure from customers and competitors, captured a strategy driven by demand from customers or pressure from competitors. The initial data analysis under this hypothesis revealed only one case where a significant association of moderate strength was found between the independent variables relating to motivation for certification and ISO related business outcomes. These are contained in Table 5 below: INSERT TABLE 5 Across all companies a moderate yet significant association was found between a continuous improvement strategy and the contribution of ISO to improved business performance in only one case. There were some significant yet weak associations within the four groups, with the strongest being in the Continuous Improvement set. This hypothesis was further tested by breaking the groups up according to the number of years they have been certified, and the results are contained in Table 6 below: INSERT TABLE 6 Analysis across the four groups by number of years certified revealed only one case of a moderate and significant association, that coming from companies newly certified in the Integrated group. These results indicate that the motivation for certification appears to not be a primary determinant of the contribution of ISO to business outcomes. There is some evidence, though, to indicate that a contribution is seen in newly certified companies with mature quality cultures. This finding appears to be consistent with those from the other two hypotheses in that companies embark on a quality journey when certifying. It would indicate that as companies move through continuous improvement processes the initial motivation for certification becomes less relevant than current attitudes. This would perhaps explain the moderate association in newly certified organisations. QUALITATIVE RESULTS Five case studies were developed on organisations that had gained certification to ISO 9000. The following is a synthesis of the qualitative findings and the lessons learned from each organisation�s experience with ISO 9000. Organisation A A number of benefits were attributed to ISO certification at Organisation A over the past six years including:

13

Improved customer confidence in the integrity of operating systems. Ability to meet the ongoing competitive requirement for quality certification. Focus on processes had led to improvements in operating procedures, standardisation of methods and ultimately better product quality. Certification acted as a catalyst for the devolvement of quality responsibility to the operator reducing the requirement for specialist quality inspectors, and ultimately increasing quality awareness at an operational level. The need for ISO certification to add value to client's systems as their quality systems and culture mature over time was identified as a potential limitation. It was identified that there is a need for a different style of audit for companies with mature systems, with perhaps the focus moving from compliance to facilitation of continuous improvement, and the role of the auditor becoming more that of consultant. The competitive context of the organisation is such that the relevance of ISO certification to continuing business performance will be an important determinant of ongoing maintenance. Organisation B A number of benefits were attributed to ISO certification over the past four years including:

• Positive changes to the quality culture of the organisation. • Standardisation provided by more structured and formal processes. • Improved systems and discipline. • Indirect contribution to changes in the industrial relations climate.

Despite these positive results, it was emphasised that ISO certification was not an end in itself, and would not provide a panacea for an organisation's quality ills. The auditor was seen to have an important role to play in ensuring the effectiveness of the system, particularly by providing continuity, flexibility and the ability to add value to the system through the audit process. The auditor also had a role to play in the ongoing training and development of the internal audit teams. This role was seen to be of primary importance in the future with the development of internal auditors being an important objective. Organisation C The fact that ISO certification acted as a major catalyst for organisational change, as well as providing a source of qualitative data on performance, were seen to be the major benefits derived from certification. Balanced against this was the potential for organisations to be limited by the minimum compliance criteria, the lack of a grading system to reward organisations for effort beyond compliance, and a perceived lack of flexibility in the standard when dealing with different types of organisations. The organisational culture required a higher level of professionalism from auditors, and this was amplified by the focus on risk identification and the incorporation of audit results into ongoing project plans. The maturity of the quality system combined with the tailoring of the ISO standard to continuous improvement activities created a requirement for an auditor able to both identify risks and provide relevant advice on projects for improvement. This organisation

14

thus had a need for an auditor able to identify opportunities for systemic improvement, rather than just check the system for compliance.

Organisation D The company has experienced a number of benefits from the implementation of ISO in the service area of their business including: marketing advantage, pro-active problem identification, a focus for induction training, the development of a documentation culture and enabling mechanism for the implementation of an intranet and a better basic understanding of business processes. Balanced against this has been the need to develop strategies for dealing with the propensity for the ISO system to become restrictive, and for the system to develop a "life of it's own". The operational environment of the organisation (ie immature growth industry in a high tech sector) determines that the auditor needs to have extensive industry knowledge in order to be effective. It was also identified that computer literacy was a key competency requirement for the auditor given that the quality system was entirely intranet based. The company also indicated that they would be prepared to pay more for an enhanced suite of services if these were offered by third party certifiers, and a possible model for these was developed with the researcher during the interview. It was also identified that the nature of the service business the company was involved in was a key determinant of the relevant auditing style to maximise benefits.

Organisation E The major benefits the organisation has derived from certification included: The ability to bid for government contracts, broader market credibility and providing a catalyst for using a system based approach to problem solving. The major limitation noted was the initial confusion encountered in deciding whether to adopt ISO into the organisational context, or to adapt the system to the requirements of the business. The appropriate role and style of the auditor, particularly in the context of a small service based organisation, was identified as being critically important to the ongoing success of certification. SYNTHESIS OF QUALITATIVE AND QUANTITATIVE ANALYSIS The quantitative analysis showed that: • There are stronger links between quality culture and benefits derived from ISO

certification in the �Performance� half of the quadrant (see fig 1). This relationship is particularly strong in organisations with developing quality cultures that have been certified for less than five years.

• The links between the style of the non-financial auditor and quality culture are stronger in the �Conformance� half of the quadrant, with no significant associations found in the Integrated grouping.

• Motivation for certification does not appear to be a strong predictor of ISO related performance.

15

The case study evidence indicated the following: The common benefits of Certification were found to be: • Customer confidence/marketing advantage • Certification was used as a catalyst for change • The Auditor acted as a change agent • Formalisation of systems and performance measures • Promotes pro-active problem identification • Catalyst for �system-based� problem solving The common limitations of Certification were found to be: • Do you adapt the business to the standard or the standard to the business? • System can develop a �life of its own� • Inability of ISO 9000 to differentiate between different types of organisations • Standards not �graded� • Relevance of ISO 9000 standards questioned as the organisation�s quality

systems mature The following possible future directions were identified: • �Cradle to Grave� rather than �Cross-sectional� auditing of processes. • The role and importance of the internal auditor expected to grow as quality

systems mature. • Grading of certified companies. • Specific auditors with skills relevant to different types of organisations. • Certification services differentiated according to the needs of the organisation. • All case study organisations stated that:

“We would be willing to pay more for a broader range of services”

IMPLICATIONS OF THE FINDINGS

Implication of the results on The Proposed Standard ISO 9001-2000 What are the implications of the findings for the new standard? This revised standard attempts to provide changes on a number of fronts: • ISO9001-2000 will offer enhanced relevance to the service sector. In light of

the findings from the case studies, this will be a positive move as organisations look to a standard that can differentiate between different types of organisations.

• Greater implementation flexibility. This again will be welcomed as another shortcoming identified from the cases has been the potential for organisations to attempt to fit their systems into ISO, rather than fit ISO to their requirements.

• ISO 9004 will need to provide a bridge between ISO9001 and enhanced quality management practices (eg continuous improvement). The research shows that as organisations move through the quality journey their certification and

16

auditing requirements change. The importance of this is highlighted by the fact that almost 50% of the surveyed organisations were in the Integrated Quality Strategy quadrant.

• An improved synergy between ISO 9001-2000 and ISO 14000 (the

environmental management system requirements) will set the stage for an organisation�s Integration of Quality (Quadrant 4 in Figure 1), environmental and other management system goals and objectives.

• The new ISO 9001-2000 will also be the foundation for the future convergence

of sectorial certification. Case study evidence also supports this as organisations look for a system able to capture their particular requirements. Over time, the trend is expected to lead toward self-declaration of conformance to a base standard, and this is supported by the findings, with mature quality organisations looking to train more internal auditors, and by extension develop a more tailored auditing service. Eventually, third party certification is likely to decrease in relevance for these organisations as quality systems mature and aim for �best practice� in continuous improvement. A real question which emerges out of this discussion, is whether international standards are so important that they cannot be left in the domain of private sector organisations to evolve.

Juran [10] perhaps captures these implications best: “The focus today should be on quality leadership-to be competitive with the rest. To achieve such leadership demands much more than CONFORMANCE to the ISO 9001 standards.

Implications of Findings for Managers More and more managers are beginning to see the value of the management systems approach to running a business ie. Institutionalising Best Practices that are derived through Continuous Improvement processes. The fact that almost 75% of organisations surveyed indicated ISO certification had contributed positively to improved business performance needs to be balanced against the fact that 60% of the sample have developed mature quality cultures. The primary implication for managers of these organisations appears to be how ISO certification can meet their evolving requirements. For example, combining the discipline of the ISO 9001-2000 compliant system with the significant process improvements that are derived from programs such as Six Sigma and cycle-time reduction could facilitate rapid process improvements that become the best practices on which to build future organisational success. The ability of the new standard to capture and meet these requirements will be a prime determinant of the extent to which managers embrace or reject it.

17

CONCLUSION Based on findings, we conclude that companies that are at the beginning stages of their quality journeys find that the ISO 9000 series of standards provides them with a guide for implementing a basic quality system and has a significant impact on operational performance. However, for companies with mature quality systems, the ISO 9000 standards were found to have less impact on business performance. Furthermore, we conclude that although conformance auditing has a role in the early stages of quality systems implementation, for auditing to be of value over the long term, it must contribute to continuous improvement of performance and system effectiveness. We believe that the revised ISO 9001-2000 standard will go a long way in expanding the scope of non-financial auditing from conformance-based to continuous improvement/performance-based. REFERENCES 1. Allan, M.J., (1993), �Implementation of ISO 9001/2 In Large Australian Manufacturers,� unpublished research project, the Melbourne Business School, the University of Melbourne, September. 2. Askey, J.M. and Dale B.G, (1994), "From ISO 9000 Series Registration to Total Quality Management: An Examination", Quality Management Journal, July, pp67-76. Association of British Chambers of Commerce, The quality infrastructure. 3. Batchelor, C., (1992), 'Badge of Quality', Financial Times, 1 September. 4. Binney, G., (1992), Making Quality Work: Lessons from Europe's largest companies, London, The Economist Intelligence Unit Management Guides. 5. Brown, A., and Van der Weile, T.A., (1996), A Typology to ISO Certification and TQM, Australian Journal of Management, Vol. 21, No. 1, June. 6. Ingman, L.C., (1991), The Quality Audit, Pulp and Paper, 65(10): 125 - 127. October. 7. Institute of Quality Assurance, (1993), Survey on the use and implementation of BS 5750 (IQA) 8. ISO 10011-1, (1990), Guidelines for auditing quality systems, Standards Australia Publishing, 1990.

9. Juran, J., M., (1999), Commentary in Quality Progress, June, pp.30.

10. Karapetrovic,S., Quality Assurance and Effectiveness of Audit Systems, International Journal of Quality and Reliability Management, Vol.17, No.6., 2000, pp.679-703.

18

11. Karpin, D., S., (1995), Enterprising Nation, Canberra, Australian Government Publishing Service. 12. Lloyds Register Quality Assurance Limited: Setting Standards for Better Business, Report of survey findings, 1993, pp1-10. 13. Miller, C, (1993) "US Firms Lag in Meeting Quality Standards," Marketing News, February. 14. Mills Charles A. (1989) The quality audit. New York: McGraw-Hill Publishing Company. 15. Russell, J.P. and Regel, T., (1996) After the Quality Audit: Closing the Loop on the Audit Process. Reviewed in quality Progress 29(6): pp65-67. 16. Science and Engineering Policy Studies Unit, (1994), "UK Quality Management-policy Options, "The Royal Society and the royal Academy of Engineering, SEPSU Policy Study No. 10, June, pp.1-99. 17. Seddon, J, (1993), BS 5750 implementation and value added (Vanguard Consulting, 1993). 18. Stratton, B., (1994), "Goodbye, ISO 9000; Welcome Back Baldridge Award," Quality Progress, August, p.5. 19. Sun, H., Total Quality Management, ISO 9000 certification and Performance Improvement, International Journal of Quality and Reliability Management, Vol.17, No.2., 2000, pp.168-179. 20. Swift, T.A., Humphrey, C., and Gor, V., Great Expectations?: The Dubious Financial Legacy of Quality Audits, British Journal of Management, Vol.11, 2000, pp.31-45.

21.Terziovski, M., (1998), �The Relationship between Total Quality Management

Practices and Operational Performance�, Journal of Operations Management,

Vol.17, pp.393-409.

22. Terziovski , M., Samson, D., and Dow, D.,(1997), 'The Business Value of Quality Management Systems Certification: Evidence from Australia and New Zealand.', Journal of Operations Management, Vol. 15, No. 1, pp. 1-18.

19

Acknowledgments

The authors wish to thank the Australian Research Council and Quality Assurance Services Pty Ltd for providing the necessary funding for this study. A special thanks to Professor Amrik Sohal for his assistance in establishing this project.

Strategy Type 4 Integrated Quality Strategy - 47% of Sample

Strong ISO 9000 with a developing quality culture

Strong ISO 9000 with a strong quality culture

Weak ISO 9000 with an immature quality culture Weak ISO 9000 with a

strong quality culture

Strategy Type 3 Continuous Improvement Strategy – 13% of Sample

Strategy Type 2 Re-Active ISO 9000 Strategy - 15% of Sample

Strategy Type 1 Pro-Active ISO 9000 Strategy – 25% of Sample

Performance

Conformance

ISO Certification Continuous Improvement

20

Regression Models (Ad. R Sq.) *= Sig.@ 5% **= Sig.@1% ALL Reactive Cont. Imp. Proactive Integrated Quality Culture ► Contribution of ISO to Improved Business Performance

0.107** 0.000 0.000 0.034 0.030

Quality Culture ► Contribution of ISO to Reduction in Waste 0.002 0.000 0.005 0.083* 0.031 Management Responsibility ► Contribution of ISO to Improved Business Performance

0.031* 0.000 0.000 0.010 0.000

Management Responsibility ► Contribution of ISO to Reduction in Waste

0.000 0.000 0.017 0.107** 0.000

21

<5 years >5 Years <5 Years >5 Years <5 Years >5 Years <2 years 2-5 Years >5 YearsQuality Culture Contribution of ISO toImproved Business Performance 0.000 0.000 0.000 0.049 .366** .205** 0.000 .103* 0.000

Quality Culture Contribution of ISO toReduction in Waste 0.000 0.000 0.041 0.000 .148* 0.083 0.000 0.061 0.015

Management Responsibility Contributionof ISO to Improved Business Performance 0.000 0.000 0.000 0.119 .337** 0.036 0.000 0.003 0.000

Management Responsibility Contributionof ISO to Reduction in Waste 0.000 0.000 0.010 0.000 .171** 0.055 0.032 0.007 0.011

Regression Models (Adj. R Sq.) - Quality Strategy - No. of Years Certified*=Sig.@5% **=Sig.@1%

Independent Variables Dependent VariablesReactive Continuous Imp. Proactive Integrated

22

ALL Reactive Cont.Imp. Proactive Integrated Auditing for Continuous Improvement Resources Provided for Quality .017* 0.000 0.000 0.000 0.005

Auditing for Continuous Improvement Quality Awareness .024** 0.000 0.000 0.000 0.000 Auditing for Continuous Improvement Benchmarking for Quality Improvement .046** 0.000 0.000 0.018 0.038**

Auditing for Continuous Improvement Customer Focus .056** .078* 0.000 0.000 0.007 Auditing for Continuous Improvement Quality as a Measure of Performance 0.00 0.004 0.000 .042* 0.000

Auditing for Compliance Resources Provided for Quality 0.019 0.000 0.069 0.000 0.016 Auditing for Compliance Quality Awareness .013* .094* 0.010 0.010 0.017 Auditing for Compliance Benchmarking for Quality Improvement 0.002 .104* 0.000 0.004 0.000 Auditing for Compliance Customer Focus 0.000 0.000 .099* 0.000 0.014 Auditing for Compliance Quality as a Measure of Performance 0.001 0.002 0.000 0.000 0.002

Regression Models (Adj. R Sq.) *=Sig.@5% **=Sig.@1%

23

Regression Models (Adj. R Sq.) – Quality strategy – No. of Years Certified *=Sig. @5% **=Sig. @1%

Reactive Continuous Imp. Proactive Integrated Independent Variables ► Dependent Variable <5 yrs. >5 yrs. <5 yrs. >5 yrs. <5 yrs. >5 yrs. <2 yrs. <5 yrs. >5 yrs.

Auditing for Continuous Improvement ► Resources Provided for Quality

0.000 0.134 0.024 0.019 0.021 0.072 0.000 0.000 0.012

Auditing for Continuous Improvement ► Quality Awareness

0.000 0.000 0.000 0.000 0.046 0.000 0.000 0.000 0.000

Auditing for Continuous Improvement ► benchmarking for Quality

0.000 0.000 0.000 0.000 0.004 0.000 0.000 .060* 0.000

Auditing for Continuous Improvement ► Customer Focus

0.012 .2766** 0.000 0.085 0.000 0.000 0.000 .059* 0.000

Auditing for Continuous Improvement ► Quality as a Measure of Performance

0.000 0.000 0.000 0.072 .192** 0.056 0.000 0.000 0.045

Auditing for Compliance ► Resources provided for Quality

0.000 0.000 .217* 0.140 0.000 0.000 0.000 0.036 0.047

Auditing for Compliance ► Quality Awareness 0.074 0.054 0.020 0.022 0.002 0.047 0.101 0.000 0.003 Auditing for Compliance ► Benchmarking for Quality Improvement

0.018 0.140 0.000 0.011 0.000 0.000 0.016 0.000 0.000

Auditing for Compliance ► Customer Focus 0.000 0.048 0.044 .249* 0.000 0.000 0.000 0.021 0.000 Auditing for Compliance ► Quality as a Measure of Performance

0.002 0.000 0.000 0.000 0.026 0.000 0.000 0.005 0.000

24

Regression Models (Adj. R Sq.) *=Sig.@5% **=Sig.@1% ALL Reactive Cont. Imp. Proactive Integrated Motivation for Certification- Continuous Improvement Strategy ► Contribution of ISO to Improved Business Performance

.128** 0.000 .082* 0.000 0.016*

Motivation for Certification- Continuous Improvement Strategy ► Contribution of ISO to Reduction in Waste

.032** 0.000 0.000 0.006 0.002

Motivation for Certification- Reaction to Environmental Factors ►Contribution of ISO to Improved Business Performance

.020** 0.000 0.010 0.000 0.004

Motivation for Certification- Reaction to Environmental Factors ►Contribution of ISO to Reduction in Waste

.008* 0.000 0.000 0.000 0.000

Motivation for Certification- Reaction to Pressure from Customers and Competitors ► Contribution of ISO to Improved Business Performance

.022** 0.000 0.000 .063** 0.044**

Motivation for Certification- Reaction to Pressure from Customers and Competitors ► Contribution of ISO to reduction in Waste

0.004 0.000 0.000 .060** 0.000

25

Regression Models (Adj. R Sq.) – Quality strategy – No. of Years Certified *=Sig. @5% **=Sig. @1%

Reactive Continuous Imp. Proactive Integrated <5 yrs. >5 yrs. <5 yrs. >5 yrs. <5 yrs. >5 yrs. <2 yrs. <5 yrs. >5 yrs.

Motivation for Certification- Continuous Improvement Strategy ► Contribution of ISO to Improved Business Performance

0.028 0.000 0.043 0.089 0.000 0.000 0.000 0.000 .049*

Motivation for Certification- Continuous Improvement Strategy ► Contribution of ISO to Reduction in Waste

0.037 0.011 0.000 0.031 0.000 0.018 .108* 0.022 0.000

Motivation for Certification- Reaction to Environmental Factors ►Contribution of ISO to Improved Business Performance

0.000 0.000 0.000 0.085 0.000 0.034 0.037 0.000 0.000

Motivation for Certification- Reaction to Environmental Factors ►Contribution of ISO to Reduction in Waste

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.010 0.021

Motivation for Certification- Reaction to Pressure from Customers and Competitors ► Contribution of ISO to Improved Business Performance

0.000 0.000 0.000 0.009 .090* .063* 0.047 0.050* 0.000

Motivation for Certification- Reaction to Pressure from Customers and Competitors ► Contribution of ISO to reduction in Waste

0.000 0.009 0.000 0.000 .077* 0.053 0.000 0.000 0.018

26

FIG 1 CONFORMANCE-PERFORMANCE MANAGEMENT GRID (Adapted from Terziovski (1998) Table 1: Table of regression models testing for association between quality culture and benefits derived from certification � All companies and the four quadrants. Table 2: Table of regression models testing for association between quality culture and benefits derived from certification �The four quadrants by number of years certified. Table 3: Table of regression models testing for association between auditing style and the quality culture of the organisation � All organisations and the four groups. Table 4: Table of regression models testing for association between auditing style and the quality culture of the organisation � The four quadrants by number of years certified. Table 5: Table of regression models testing for association between motivation for certification and contribution of ISO to business performance � All companies and the four quadrants. Table 6: Table of regression models testing for association between motivation for certification and contribution of ISO to business performance � The four quadrants by number of years certified