The Evidence for Continuous Quality Improvement:

A Literature Review

October 2009

Prepared for the QA&CPD Sub-Committee by Peter Stephenson, QA&CPD Project Officer

Contents Contents .........................................................................................................................1 Introduction....................................................................................................................3

Aim ............................................................................................................................3 Method .......................................................................................................................3 Study Limitations.......................................................................................................3

Quality in health care .....................................................................................................4 What is quality? .........................................................................................................4 Background ................................................................................................................4 A quality crisis-major service failures .......................................................................5

Improving the quality of healthcare...............................................................................6 Education ...................................................................................................................6

CME.......................................................................................................................6 CPD........................................................................................................................8

Quality Assurance and Accreditation ........................................................................8 Performance or outcome measures ............................................................................9 Financial incentives .................................................................................................10 Clinical governance .................................................................................................12

CQI...............................................................................................................................13 Origins......................................................................................................................13 What is CQI?............................................................................................................13 Adoption of CQI for use in health care....................................................................13 Does CQI work? ......................................................................................................14

GP examples ........................................................................................................16 Collaboratives model ...........................................................................................16

The evidence says… ................................................................................................18 What hinders or helps?.............................................................................................19

Barriers.................................................................................................................19 Enablers................................................................................................................21

What tools should we use to evaluate CQI? ............................................................23 The paradigm of cause and effect vs complex adaptive systems.........................24 A fresh look at QI research ..................................................................................26 Why the lack of available evidence? ...................................................................27 Publication bias....................................................................................................28

Is it research or CQI? ...............................................................................................29 Need to combine interventions ................................................................................29

Conclusions..................................................................................................................30 References....................................................................................................................31

Introduction

Aim

This paper presents findings from an in depth literature review conducted by QA&CPD project staff during August and September 2009. The review was aimed at exploring the efficacy of continued quality improvement (CQI) in health care broadly, and medicine and general practice more specifically. The study was bounded by the following abstract:

Growing evidence from around the world shows quality improvement (QI) activity does lead to practice change for the better, and that an emphasis on greater interprofessional cooperation in the primary health care team setting, based around general practice, is a more effective health care solution than other forms of care in the secondary and tertiary sectors.

Method

This study has drawn widely from the full range of international and local peer reviewed and scholarly journals available through the RACGP and other electronic databases. Limited grey literature has also been examined, particularly focusing on relevant government reports. The majority of works drawn upon have been peer reviewed.

Searches have been conducted using a number of key words and phrases drawn from and similar to the themes expressed in the abstract. Further articles cited in reference lists of particular value to the study have been examined and included where appropriate. Approximately 400 articles and texts have been consulted, with less than half of those providing relevant material.

Study Limitations

While the authors have sought to provide a comprehensive and balanced view of the extant literature, there has to be a limit to the time that can be dedicated to a project such as this. Where documents were not readily available, a judgement call was made on the basis of the information contained in the abstract as to whether the document was critical and efforts were made to obtain it externally via the College library services or other means. In the interests of time, those not deemed critical were not pursued further. Therefore, there is the likelihood that we have missed some literature that may have contained relevant content. We would believe however, that any such exclusion would be unlikely to change the conclusions that have been drawn and that others may draw from the reading of this paper.

Quality in health care

What is quality?

Quality in health care has been variously defined. An early paper by Moss et al. suggested that quality in health care is usually understood in terms of clinical quality “…and an implicit distinction is drawn between managerial and clinical activity.”(1) Clinicians may define quality as “doing the right things, for the right people, at the right time, and doing them right first time.”(2)

According to Wilson et al., quality is an abstract concept and a number of aspects of care need to be considered to gain a picture of quality in health care. These include technical quality–doing the right thing and doing it right; interpersonal quality–how the patient felt they had been treated(3); and costs of care. (3, 4)

Larson and Muller(5) and others(6-9) cite the seminal works of the 60s and 70s of Avedis Donabedian who viewed quality of care in terms of ‘structure’, ‘process’ and ‘outcome’. Structure refers to the setting in which the care takes place, which includes the qualifications of staff, organisational structure, and policies and operation of programs.(5, 6, 9) Process measures refer to the technical management of illness but also include rehabilitation, prevention, and continuity of care and aspects of patient physician interaction.(5, 6, 8, 9) Outcome of care is defined relative to recovery, restoration of function and survival.(5, 6, 9)

Traditionally the assessment of the quality of patient care has been left to health professionals through formal peer review and quality assurance programs.(10) Little attention has been paid to the perspectives of consumers.(11) Some argue that patients cannot(12) or don’t feel qualified to judge technical quality(13, 14). They do however, have a legitimate and valuable perspective on the care they receive(10, 14) and judge their care on measures that they personally value(13).

While Pegram and Wright argue that quality is difficult to define unambiguously, in Australia the Commonwealth Department of Health and Family Services have adopted(15) the Institute of Medicine’s definition: “Quality is the degree to which health services for individuals and populations increase the likelihood of desired outcomes and are consistent with current professional knowledge”.(9)

Background

Early assumptions about the quality of health care were based on the premise that if you built the right infrastructure(11, 13) and provided the appropriate education and training for staff, quality outcomes would result.(6, 11, 13) “Quality was seen as inherent in the system, sustained by the ethos and skills of the health professionals working within it”.(2) These early concepts of quality were focused on competence with particular emphasis on the individual practitioner.(6, 16, 17) As a consequence, quality initiatives were largely based on education and educational interventions, and stemmed from the principle that quality flowed from education.(16)

Despite the good intentions of physicians and other health professionals(11), for some time it has been apparent that reliance on such an approach could not guarantee quality(18); particularly with continued advances in medical technology(6, 19), growing costs(6), the demands on physicians to remain up to date with the sheer amount of information(20-22), and the expectations of governments(23) and patients.

A quality crisis-major service failures

Over the past two decades, a series of well publicised reports of health service failures have brought the issue health care safety and quality to the fore. In the United Kingdom, a number of hospital failures(13, 24-26) have been “…of such seriousness that they have resulted in major inquiries”.(25) In the United States, studies conducted by the Institute of Medicine highlighted patient safety; suggesting as many as 98,000 people may die per year due to medical errors(27); and quality of care; noting that many patients do not get the care they deserve.(28)

Overuse, underuse, and misuse of medical care are major contributors to the gap between average quality of care and the best that is available.(28, 29) While Shortell et al. claim that up to one quarter of hospital deaths may be preventable, they also suggest that one third of some hospital procedures may expose patients to risk without improving their health and again, one third of laboratory tests showing abnormal results may never be followed up by physicians.(30) More recent research conducted in the US found “…widespread errors, inefficiencies and missed opportunities for improvement in Australia, Canada, New Zealand, the UK and the US.”(31) Shortell et al. suggest that these outcomes may worsen due to increased pressure to contain costs. (30)

In the Australian context, ‘The quality in Australian health care study’ (AQHCS)(32) conducted in 1995 alerted to concerns for the safety of patients in the hospital system. Following on, a series of “…sensational scandals involving patient care…” emerged in various states in Australia before the 2005 Bundaberg hospital scandal. (33)

Van Der Weyden wrote in 2003: “All is not well with Australia’s health system”, and “…(is) cracking under the strains of a growing mismatch between its capacity to deliver quality healthcare and the changing demands of our communities.”(34) There exists however, a “…depth of desire for healthcare reform among consumers and professionals”.(34) Indeed, it appears there is little doubt that most physicians are highly motivated to improve quality of care(18, 35-39), however, such motivation alone won’t improve patient outcomes(37).

The most common causes of sub optimal health care are breakdowns or failures of the systems of care(10, 31), which will require a systems approach to address.(31, 40) Booth et al. state that most errors result from problems created by today’s complex health care system.(10) Whilst the Bundaberg example involved the practice of the individual physician, it was still a systems breakdown that led to the issue.(33)

Improving the quality of healthcare Despite the concern raised by the major reports into patient safety and health(27, 28), the ‘quality revolution’ among healthcare organisations in the US and elsewhere has “…not as yet been consistently associated with higher levels of service quality.(7) Solberg et al. question whether there has been any improvement in healthcare with the exception of reducing waits and sometimes harmful delays.(41) In a later paper, Solberg states that while there has been some slow improvement in care quality–particularly for some chronic conditions–the reports commissioned by the Institute of Medicine have not led to the dramatic improvements called for.(42)

The pursuit of quality in health care has a long history, one which has incorporated a wide range of approaches. These include education: to ensure the skills and knowledge of physicians and other health care professionals; and philosophies and programs designed to variously assess, audit and improve all aspects of the systems, processes and delivery of care; with the ultimate aim of providing the best possible patient outcomes. Says Oxman et al., “There are no ‘magic bullets’ for improving the quality of health care, but there are a wide range of interventions available, that if used appropriately, could lead to important improvements in professional practice and patient outcomes”.(43) The following pages will describe some of the more common interventions in more detail.

Education

As earlier indicated, education has long been seen as integral to the delivery of quality outcomes in health care. It was understood that quality outcomes would result from the provision of the appropriate education and training for staff. However, while proper training is vital to assure the quality of health care, such training is not sufficient alone.(6)

While educating physicians and other health professionals has taken various forms, such activities are generally categorised as continuing medical education (CME) or continuing (or compulsory) professional development (CPD).

CME

Traditional continuing medical education (CME) has been underpinned by a belief that gains in knowledge in turn lead to practice change and subsequently, to improved patient outcomes.(44) In recent decades, that assumption has been strongly challenged (44-49) and calls for revamping of CME have been prevalent.(45, 47, 49-51) Despite some evidence that CME may increase physician knowledge(29), didactic or passive forms of CME (for example conferences, lectures and handouts) in particular, are unlikely to contribute to practice change(22, 44-47, 52, 53); thus failing to lead to improved patient care(45, 47, 54) or outcomes(44). The inability of conventional CME to impact on improved patient care is further highlighted when CME becomes a quest for credit hours or points to fulfil requirements.(55) As a ‘stand alone’ strategy for ensuring practitioner competence, traditional CME is insufficient.(56, 57)

It is now understood that for maximum benefit, learning needs to be active rather than passive.(20, 50, 58) Traditional forms of CME have done little more than count attendance(45, 50, 51), whereas its aim should have been to promote and recognise “…physician achievement in knowledge, competence, and performance”.(50) It is not surprising that passive forms of CME have persisted however, as they are cheap to run(44), can accommodate large numbers of participants(45), require relatively little skill and preparation on behalf of the presenter(44, 45), and are attractive to potential sponsors.(44, 45)

According to Nahrwold, CME also needs to be both practice specific and focused on the needs of physicians’ patients.(55)

Rewarding the physician for episodes of learning that improve the health and well-being of his or her patients will be a powerful stimulus for the physician to do just that.(55)

More active forms of CME have been more successful in effecting practice change(53), some of which can leave benefits lasting several years.(40) Peer group learning is among those strategies which have yielded success.(59) Armson et al. concur:

Interactive approaches can be effective, particularly when they involve small peer groups that foster trust, promote discussion of evidence of real cases, provide feedback on performance, and offer opportunities for practising newly acquired skills.(22)

Academic detailing–a term used to describe brief one-to-one interactions between primary care practitioners–has been found to be effective at changing practice as well as improving patient outcomes.(60) Boom et al. reported on a study which used a similar process of using educational outreach visits from peers, in seeking to change behaviour among all levels of practice staff. Significant improvement was found across all types of practices included in the study; and across a broad range of practice activities such as maximizing immunizations due, screening immunization records at every visit and improved vaccination intervals, although it should be noted that results cannot suggest direct causality. (61)

Van den Hombergh et al. note that in combination with other interventions, educational outreach visits have proved to be an effective QI strategy. They claim that this approach proved particularly effective in reducing over prescribing.(62)

In their review of the evidence for electronic or web based continuing education initiatives in the health professions, Lam-Antoniades et al. found that such initiatives have grown exponentially in the past decade and offer many advantages including easy access, low costs, flexible timing and the ease of adaptability for individual learning styles. They add that some evidence exists to suggest that internet based CME programs were equivalent to traditional methods in achieving knowledge change, however, little information is available on whether this extends to practice change.(63) One of the studies reviewed by Lam-Antoniades et al. showed that much of the literature in relation to the outcomes of web based CME was based on participant satisfaction ratings and limited data existed to

show changes in clinical practice, while no data demonstrated changes to patient outcomes.(63)

CPD

In contrast to the content-focus and a teacher-centred approach of traditional CME, CPD adopts a more learner-centred approach(49), focusing on the individual learning needs of physicians.(49, 64) Incorporating the principles of adult education(65) including self-directed learning and reflective practice(52, 64), learners therefore play the pivotal role in the development of their own education. Davis et al. have found that participating in this type of interactive, education has the potential to improve physician performance and patient outcomes.(44)

More holistic than traditional CME, CPD extends beyond the clinical domain and may impact on the entire spectrum of professional activities of physicians including practice management, leadership, administration and education.(49)

Quality Assurance and Accreditation

One early approach to managing quality in health care has been described as quality assurance (QA). Formal approaches to QA have a long history. As early as 1858, the pioneer English nurse Florence Nightingale, famous for her work in the Crimean war “…was concerned about the outcomes of hospital care…” and implemented processes which improved mortality rates. Similarly A.E. Codman, a US surgeon in 1917 began measuring hospital patient outcome data with the aim of understanding and improving patient outcomes.(6)

Essentially, QA is a formal and systematic approach to problem identification and eradication. Described by Berwick as the ‘Theory of Bad Apples’(66) in its earlier forms, QA was largely been seen as a model that identified errors after the fact (14, 67)and sought to apportion blame on the professionals involved.(14, 64, 66)

“Traditionally, in North America, the term QA has been associated with retrospective review of medical records to assess the quality of care (sometimes referred to as medical audit)”. In Australia, Wilson and Goldschmidt argue that “…the term quality assurance has been applied inappropriately to any kind of review of any aspect of clinical or non clinical activities”. They prefer a definition that includes the PDSA cycle and “…denotes a cyclical process involving retrospective quality assessment…”, (based on the patient’s medical record) and subsequent steps to improve care processes to close any performance gaps.(6)

Using tools such as accreditation, clinical audits (a QI process which compares actual clinical practice against established standards of practice)(68), practice guidelines (which make explicit assumptions between health care processes and patient health status improvement)(6), and checklists(11, 17), QA in health care has sought to ensure that “…care conforms to specifications and to provide information to validate and improve them”.(69) It includes activities such as the monitoring and tracking of incidents; analysis of complaints and compliments; infection control; and sentinel events such as unexplained deaths, readmissions

within specified time frames, and unplanned admissions to intensive care units (ICUs).(6)

According to Wilson and Goldschmidt, QA is “…largely something that has been imposed on hospitals and doctors by regulators and governing bodies”. They add that externally driven and internally motivated QA efforts have been minimal in Australia.(6)

Accreditation has been a widely employed tool in health care in an effort to bring about QI. In the US, external monitoring agencies have undertaken quality assurance using accreditation since the 1950s.(70) Involving measurement against a set of standards(6), accreditation is now virtually mandatory for those hospitals wishing to be eligible to receive Medicare and Medicaid funding.(71)

In the Australian context, the Australian Council of Health Care Standards developed standards for hospital accreditation in Australia in the 1970s and 1980s and shortly after, the Australian Government made the decision to align various government payments to general practice accreditation standards.(72)

As to the effectiveness of accreditation as a QI initiative, Buetow and Wellingham highlight the importance of not simply meeting or exceeding the minimum standards, but striving for excellence.(73) According to Al Tehewy et al. however, there is little conclusive evidence that the accreditation process actually improves the quality of care offered in health services. They state further that “…reasons for such a lack of evaluation research may include the methodological challenges of measuring outcomes and attributing causality to organizations or health systems, which themselves are complex and changing”.(74)

O’Leary notes that “Although couched in the language of continuous quality improvement, the accreditation process is, at its core, a risk reduction activity”.(75) Wilson and Goldschmidt hold a different perspective. They view accreditation as a valuable QA tool, and QA as an essential mechanism for CQI: “…QA and QI are important but distinct aspects of production and product quality; CQI is merely performing QA/QI on a continuous basis”.(6)

Performance or outcome measures Published health outcomes are often seen as the measure of how an organisation is performing. Used appropriately, they may also be useful tools to shape service delivery and prompt quality improvements.(76) However, aside from the obvious such as death, post operative loss of a limb or diabetic coma, many outcome measures in health care are difficult to administer and measure, and often rely on subjective assessments of a patient’s status.(8) “Measurement alone does not improve quality…”Sheldon(4)

Conversely, most process measures can be readily measured and interpreted. (8) (8) Further, studies of process can be more effective at detecting poor outcomes than outcome studies, and often point directly to the actions needed to improve care.(8)

There is a view that performance measurement can have some negative unintended consequences(4, 77) including ignoring unmet need, and manipulation of data.(4) Sheldon suggests that the use of indicators is extending to areas that have only a small clinical benefit and may risk diverting attention to more important but unmeasured aspects of care. “Thus while the measured performance may improve, quality may fall in the less scrutinised areas, resulting possibly in a fall in overall performance”. (4) Johnson gives the example of foot inspections of diabetes sufferers being weighted equally with achieving glycemic control.(77)

In addition, while quality indicators may break down the process of care into measureable elements, not all of these elements are of equal importance to the patient, the provider, or the community.(77) Good performance is not necessarily good care and pressure to improve performance may be at the expense of good care.(78) And further, an indicator may suggest that an episode of care may be quality, yet it has not met the individual patient need.(78)

Kapur suggests that other factors need to be considered when designing outcome measures of care, including the patients’ participation in everyday activities that they were able to participate in before the episode of care(79), and the degree of stress of the patient and the well being of family members; and the satisfaction of the patient and families with the episode of care. (79).

Further, if the use of such measures leads to a culture of blame, are used in isolation of improvement interventions or strategies, or ignore whole of patient care at the expense of targeted measures, quality and accountability of care may suffer.(4, 77, 78) Failure to deliver high quality care in areas that are not being measured at the expense of the measurable may increase as performance measurement is linked to financial or other incentives.(78)

Performance measurement does have a role in primary care toward improving quality and promoting accountability. At a practice level, measures give providers specific feedback on how they compare with peers in achieving particular quality targets.(77) Good performance measures encourage physicians to adhere to guidelines for which there is strong evidence that improving practice will improve patient outcomes.(78)

Financial incentives

Changing physician behaviour is a key to improving quality of care(38), however, such change is notoriously difficult to achieve.(80) Financial incentives have been suggested as a way to affect clinician behaviour toward better quality care, though information on how best to design such programs and understanding of potential impacts is limited.(38)

Pay for Performance programs are gaining currency at all levels of the health system as a quality improvement initiative, with the hypothesis that money will change behaviour.(38) This hypothesis is contingent on physician engagement and one study reported by Teleki found that physicians exposed to financial incentives were not engaged and did not change their behaviour in response.(38)

On the other hand, and despite limited evaluations of incentive schemes(38, 81) a recent study of chronic disease care carried out in New York suggested that financial incentives for primary care physicians did lead to improvements in the objective quality of care measures.(81)

Gosfield argues that much of the focus on compensation and revenue for physicians is not directly about money. Rather, it is about the loss of time in their lives and how that affects their ability to provide high quality care.(82)

There is a view that financial incentives may lead to unintended consequences such as described as ‘creaming’–focussing on patients likely to provide positive performance scores, and ‘dumping’–removing from patient lists those who negatively affect performance scores.(6) However, a study by Teleki found that nearly three quarters of physicians did not believe that financial incentives would have such an effect.(38)

A further issue in relation to performance based incentives is about the autonomy of physicians. It is argued that a guideline driven process of care may leave physicians “…feeling that they have become less skilled, are losing their sense of place in the clinical enterprise, and are less connected with patients”.(80)

According to Galvin, a dichotomy exists in relation to health care and quality. For example, in the US (where health care is provided through insurance by employers or health plans) if overuse is reduced, one aspect of quality improves. In this case, the employers and insurers are the winners, while the provider losers.(83)

Shine similarly gives the example of a quality improvement initiative based on an enhanced communications system with patients. This initiative helped patients receive better care, had better outcomes and required fewer doctor visits. But because these doctors were reimbursed on a fee for service basis and patients didn’t return as often, it was costing these doctors to improve quality.(29)

A further example of this dichotomy described by Shine is hospital which successfully used a QI program that cut mortality rates and hospital days and reduced costs. however, because of the funding systems in place, it cost the hospital money.(29)

In the UK, a payment incentive scheme for GPs is the Quality Outcomes Framework (QOF). According to O'Brien et al., the QOF does not support many aspects of quality development.(84)

While financial incentives may have their place in the broader push for quality improvement, how to best structure such schemes effectively to improve quality is the key question. Says Teleki, financial incentives are unlikely to be a ‘magic bullet’ to improve quality of care”.(38)

Clinical governance

In the UK, since the reporting of the failures within its hospital system, clinical governance has been introduced as a mechanism to help develop safer systems. Clinical governance can be seen as an overarching framework; a whole system of cultural change which provides the means of developing the organisational capacity to deliver sustainable, accountable, patient focussed, quality assured health care(11) and is arguably a model that “…may be transferable to other settings or organizations”.(85)

Franks suggests that “…as an encompassing principle, clinical governance is no more, and no less, than a process of focused and continuous quality improvement with explicit duties of accountability…”(86) While there is broad agreement that clinical governance has an emphasis on CQI in health care, there exist many organisational interpretations.(24)

According to Swerrison and Jordan (citing Sweeney, Sweeney, Greco and Stead 2002) a study in the UK within primary care found a number of positive impacts of clinical governance at a practical level, including the consistency of standards, consistency in data collection, better service provision, better team work at a multi disciplinary level, a reflective and proactive culture, and better treated patients. (87)

Australia has also embraced the notion of clinical governance.(11, 87, 88) The Australian Council of Healthcare Standards (ACHS) defines clinical governance as:

the system by which the governing body, managers and clinicians share responsibility and are held accountable for patient care, minimising risks to consumers, and for continuously monitoring and improving the quality of clinical care”.(87)

Nonetheless, despite improvements in the decade since the QAHCS report, Wilson et al. argue that there are still four areas requiring “…more action and greater urgency”. These include leadership, transparency in reporting of quality of care, measurement (information and data is needed to guide improvement efforts and measure effectiveness of interventions) and improvement tools.(89)

What is also clear is that just as in industry, to achieve quality in health care will requires the systems to be planned, organised and managed.(3) Improvement in quality of care will not simply happen by the good intentions of health professionals, no matter the level of their skills, training or knowledge.(3, 90)

CQI

Origins

Continuous quality improvement (CQI), also known as Total Quality Management (TQM), emerged from the revolutionary work of Shewart and later; Deming, Juran and others in redesigning quality processes in industry.(1, 6, 16, 67, 69, 91, 92) Described as an effective package of theory and practical tools to reduce errors in the production process(92), this management approach was employed with notable success in post WWII Japan.(1, 6, 93) During the 1980s(69, 94) these ideas began to be employed beyond their manufacturing origins into health care.(6, 30, 67, 69, 91, 94)

What is CQI?

A number of definitions exist for CQI and TQM however, they are commonly used interchangeably.(6) Shortell et al. state “The key elements in a combined definition of CQI/TQM include continuous improvement, customer focus, structured processes, and organization-wide participation.”(95) Weiner et al. define CQI as “…an ongoing effort to provide care that meets or exceeds customer expectations”.(96) According to Counte et al. “CQI can be defined as a customer-driven leadership approach based on the continual improvement of the processes associated with providing goods or a service.”(7)

Based on engineering principles, specifically systems theory, CQI is as much associated with the physical sciences as the social sciences. It is not just about analytical methods, but also organisational factors that may hinder or support implementation.(67) The CQI model has the potential for wide application for ongoing improvement and employs the familiar ‘PDSA’ cycle of four continuous steps: ‘Plan’, ‘Do’, ‘Study’ and ‘Act’(9, 68, 90, 97), (previously PDCA: plan, do, check, and act).(6)

Continuous Quality Improvement uses a common set of quality improvement tools and techniques in each stage such as cause and effect diagrams(69, 98), also known as fishbone or Ishikawa diagrams(69); process mapping and flowcharting. Through the use of such tools, CQI enables the systematic evaluation of processes, identification of problems within processes and improvement of processes throughout the lifecycle of the product or service; rather than at the end of production as would be the case with quality assurance.(67)

Adoption of CQI for use in health care

Since the late 1980s, there has continued interest in CQI across the US health care system(7) with cost being the main driver for change.(69) According to Swinehart and Smith, the US accreditation body (Joint Commission on Accreditation of Health Care Organizations [JCAHO]) recognised that while quality assurance programs do provide improvement, such improvement is not necessarily felt equally across the organisation. The CQI approach was seen to be the viable

solution to improving the quality of health care (99) and has been the approach most “…often, and visibly…” adopted.(30)

Outside of the US, CQI has been embraced in the United Kingdom and in countries including Canada, Australia, New Zealand and the Netherlands.(7, 100) In the US, a driver for the introduction of CQI has been largely market based, whilst in other countries, the impetus has been more linked to government regulatory involvement.(7)

Does CQI work?

Strong evidence exists to suggest that TQM/CQI programs have been successfully employed in a number of industries.(7, 69, 99) In health, reports of such success are fewer. There exists a number of views as to the efficacy and overall value of the TQM/CQI approach to quality improvement.

According to a Blumenthal et al. study, while there are a range of success stories in industry such as Ford, Harley-Davidson, Xerox, and Motorola (cites Womack et al 1991), CQI has not had the impact hoped for and as had been seen in other industries. Further:

…none of the national quality experts interviewed for this study could identify a health care organization that has fundamentally improved its performance by CQI or any other means.(14)

They add that many executives were unable to provide quantitative data resulting from CQI projects in their organisations, rather, they told anecdotes to support their conviction that many projects had led to improvements in processes or outcomes of care.(14)

It appears that some organisations which attempt CQI are successful and some are not.(101) Clearly, the success of quality improvement initiatives within the medical profession and the health care sector more broadly is the subject of considerable debate. According to Blumenthal et al. “…the health care system still knows little about the efficacy of much of that it does.”(14)

Weiner et al. cite a number of commentators who believe that the systematic application of industrial quality improvement (QI) methods can result in a significant improvement in clinical processes and medical care outcomes.(96) In their 1998 work, Shortell et al. suggest that evidence exists that quality and outcomes of care can be improved through CQI.(30) Indeed, the literature on CQI has produced some evidence–albeit based on nonrandomized studies–that it’s clinical application can improve patient outcomes, whilst reducing costs.(30)

Salman reported improved quality of care of patients with diabetes and patients with hypertension as a result of a quality improvement process in a rural health clinic.(102) According to Ferris et al. “Published reports indicate that substantial improvements in the care of children can be made through QI”.(103) In their study on the use of CQI to improve patient outcomes in the area of vascular access planning, Barton et al. found that the CQI process can be used successfully to positively impact patient outcomes through process improvements with complex critical problems.(104) Reporting on the findings from a quantitative research

study of quality management in the Irish health-care sector, Ennis and Harrington suggest that “…the impact from the introduction of quality is promising with improved patient satisfaction and improved quality awareness being the most predominant factors”.(105)

Swerrisen and Jordan cite a 2002 UK study within primary care which found a range of positive impacts at a practical level, including consistency of standards and data collection, better service provision, better team work at a multi disciplinary level, a reflective and proactive culture, and better treated patients.(87) Continuous quality improvement methods have also been used successfully by a multidisciplinary team; including nursing staff, physicians, health records personnel and a CQI facilitator; to achieve a sustained reduction in induction rates in a Canadian maternity hospital.(106)

In 1998 Chassin et al. wrote that across the health care industry, “…quality improvement efforts were sporadic at best and typically limited to single large institutions such as hospitals”. They added that long term multi institutional quality improvement programs are infrequent and interventions meant to improve outcomes across entire delivery systems are rare. They did however, highlight several noteworthy exceptions where state-wide mortality rates had been lowered as a result of the interventions.(92) Eagle et al. report the use of a rapid-cycle quality improvement effort across 33 hospitals leading to indirect process measures of care, “…that translate into improved patient outcomes”. The same study suggests the quality improvement effort is associated with a lower 30-day and one-year mortality rate among Medicare beneficiaries hospitalized for an acute myocardial infarction.(107) In their evaluation of an initiative implementing 60 CQI projects in French hospitals, Maguerez et al. found that feedback “…indicated that the CQI initiative met with some success despite some difficulties, especially during the initial stages”.(108)

Lanier et al claim that in the US many examples of continuous quality improvement processes have shown sustainable performance improvement under rigorous assessment(17). Other reports of successful interventions include the use of continuous quality improvement methodology to improve prescription turnaround times and reduce practice costs.(109)

A study by Fox aimed to evaluate a multi clinic program aimed at improving asthma related health outcomes in schoolchildren reported major outcomes in care processes and clinical processes.(110) According to Greenfield–in spite of the Fox study not meeting the standards for asserting causality–the evidence suggests that the proposition that the observed improvements were related to the intervention was supportable.(111)

Examples of some success are found also in the developing world. Mohammadi et al. report that most respondents to an evaluation survey reported positive impacts on organisational culture, work efficiency and quality of services, following the implementation of a formal quality improvement program in a teaching hospital affiliated with the Tehran University of Medical Sciences.(112)

GP examples

In their report of the transformation of a run down general practice, Jackson and Bircher found the EFQM excellence framework based on the CQI ‘PDSA’ cycle is easily applied to general practice. Stating that the ultimate test is whether it makes things better for patients, they argue that before they began, “…the team would have said ‘we think so’, while after implementation they can say, ‘we know it does’.”(113)

Other reported successes include a Canadian family practice quality improvement initiative aimed at improving patient care by facilitating access to timely appointments,(114) and a study focusing on achieving improved clinical outcomes through an interdisciplinary quality improvement intervention. Results of that study showed that nine out of the 25 interdisciplinary teams involved were successful in improving patient care and outcomes through implementing practice change.(115)

Verstappen et al. compared the cost effects of two quality strategies to improve test ordering in primary care using a randomized trial. The study found costs savings to the intervention ‘arm’ of the study while other non monetary benefits may flow from such quality improvement initiatives including increased physician knowledge. Further, there is some empirical evidence that participating in such quality improvement activities may lead to increased physician job satisfaction.(116)

Ionnidis et al. conducted a pilot study into the use of ‘Quality Circles’ to improved physician adherence to clinical guidelines in Canada. While not claiming improved clinical outcomes, the multifaceted pilot project found the “…use of QCs feasible for transferring knowledge and give physicians an opportunity to analyse work related problems and develop solutions to them.”(117)

Collaboratives model

A key vehicle for introducing and sustaining quality improvement initiatives in Australia and abroad is the Quality Improvement Collaborative (QIC). “Collaboratives are similar to hospital quality programmes in that they usually involve project teams, but the teams are from different organisations.”(118) Mittman describes QICs as “… an approach emphasizing collaborative learning and exchange of insights and support among a set of health care organizations.”(119)

Despite the growing popularity of the collaboratives approach, there are questions about its efficacy:

Despite limited evidence, the quality improvement collaborative is one of the most popular methods for organising improvement efforts at hospitals and ambulatory practices worldwide. (120)

Mittman also questions a dependence on the QIC approach to improvement:

Unfortunately, the widespread acceptance and reliance on this approach are based not on solid evidence but on shared beliefs and anecdotal affirmations that may overstate the actual effectiveness of the method.(119)

Further, Mittman states that while numerous reports of collaboratives have been published, they lack objective impact evaluations, consisting primarily of subjective or self report assessments of QIC effects and qualitative summaries of key ‘lessons learned’ by those involved.(119)

A view exists that the overall effectiveness of the QIC method remains highly uncertain but is probably modest (119, 121) and others express scepticism regarding the benefits of the QIC method, citing “…highly variable results, inconsistent approaches and generally small sample sizes.”(122) Others conclude that the evidence in favour of the use of quality improvement collaboratives while positive, is limited (100, 123) and their effects are difficult to predict.(100)

A study conducted by Landon et al. to evaluate the effectiveness of a quality improvement collaborative in improving the quality of care for HIV-infected patients found that the collaborative did not significantly affect the quality of care.(121) Mittman suggests that the Landon study is more rigorous than previous studies into the effects of collaboratives and,

...demonstrates small and generally insignificant pre-post improvements among both intervention and control sites in most of the quality indicators measured…(and) is high quality, objective evidence that helps break the cycle of belief about the QIC method. (119)

Biuso and Newtons’ 2008 article reports on the successful use of the collaboratives approach to supporting the treatment of chronic disease in the UK and suggest that its use in the Australian general practice environment has similarly “…seen some significant improvements across each of the topic measures.”(124) Other collaboratives are similarly reported to have had some success (120, 122, 125) including a study of one collaborative which showed that after 18 months, dramatic improvements were seen in several key process of care measures, but little headway had been made in others.(120)

In their 2006 evaluation of a learning collaborative aimed at improved delivery of preventive services by paediatric practices, Young et al. found evidence that motivated practice teams can learn and apply easily basic improvement methods and when they do, that the quality of care they deliver seems to improve. They added that while the study showed an overall improvement in preventative services delivery, some of the increases were modest, though statistically significant. (122)

Schonlau et al. reported that a collaborative implementing the Chronic Care Model to improve asthma care had a positive effect on patient self management practices that have previously been linked to improved health outcomes. They add that follow up of patients who participated in the intervention may have been too brief to be able to detect significant improvement in health related outcomes. (126)

According to Landon et al. their study, like other evaluations of QI programs, “…is generalizable only to the disease we studied, a chronic medical condition.” They suggest that different clinical areas might be more amenable to the improvement methods studied and note a need for additional research to improve methods of teaching and implementing QI improvement projects to obtain better results.(121) Likewise, Solberg states that collaboratives are not homogenous: “…if you’ve seen one QIC, you’ve seen one QIC”.(127)

Quality improvement collaboratives seem to play a key part in current strategies focused on accelerating improvement, but may have only modest effects on outcomes, at best. Further knowledge of the basic component effectiveness, cost effectiveness, and success factors of QICs is crucial to determine their value.(100)

The evidence says…

Randomized trials testing the effectiveness of CQI as a quality improvement approach in medicine have to date produced mixed results.(128, 129) Ferguson et al. write of a randomized controlled trial of the use of continuous quality improvement to increase use of process measures in patients undergoing coronary artery bypass graft surgery. This trial had moderate success in relation to improvement in the use of the measures. Accomplished nationally, the trial was rapid compared to many CQI efforts.(128)

We have some empirical evidence but this is often limited to benefits without reference to costs and lacking any comparative component; and some experiential evidence, which is often from the most enthusiastic CQI early adopters and pioneers. (98) Walshe adds that we rarely have much theoretical evidence and “…research suggests that the impact or effectiveness of healthcare QI programmes is often rather variable and limited.”(98)

While some interventions may have failed to deliver the direct and specific benefit improvement being measured, there are studies which presume benefits nonetheless. For example, one cluster randomized trial conducted to assess whether completing an asthma-specific PIM resulted in improved patient outcomes found that the primary outcome of dispensing inhaled corticosteroid (ICS) after a post-intervention visit was not improved, though it may have lessened asthma severity through an increased discussion of asthma triggers.(130)

In discussion of a randomized control trial of a US national QIC, Solberg concurs with Mittman’s earlier conclusion that the overall effectiveness of the QIC method “remains highly uncertain but is probably modest”(119), “…but only for that particular type of QIC, which is national, selective and focused on a single topic.”(127)

In the Australian context, Bailie et al. attributes improvements in Indigenous primary care systems to a continuous quality improvement based approach (131) while a Dutch project aimed at assisting the decision making of GPs in dealing with patients at high risk of cardiovascular disease improved the “…clinical decision making for some aspects of clinical care…”. While the overall effects were considered small, the authors believed that the intervention may also ultimately “…improve patient outcomes.”(132)

A 2005 project to improve the early detection of cancer in primary care saw five health organisations provided with funds to undertake continuous quality improvement activities aimed at improving the early detection of cancer within each organisation or pilot site. Each organisation noted improvements in early cancer detection and improved screening results however, causality could not be demonstrated.(133)

Brindis et al. report improvements in patient outcomes in cardiovascular care but suggest that because theirs was not a randomized study, it was not possible to attribute causality to the interventions applied.(134) Other authors also suggest that study design precluded causal inference, citing the difficulty of determining “…which component(s) of the interventions were responsible.”(135) and the potential for concurrent events or temporal trends to influence results.(41)

According to Buetow and Coster, the effectiveness of initiatives for continuous quality improvement is questionable(136), while a “…more sceptical and scientifically rigorous approach to the development, evaluation and dissemination of QI methodologies is needed.”(98)

The peer-reviewed literature demonstrating improvements in quality from CQI activities is not ‘voluminous’ and is particularly sparse in clinical papers in prominent scientific journals demonstrating improvements in clinical outcomes.(14) According to Mittman, the evidence to date is insufficient to reject the null hypothesis of no overall effect.(119) Walshe too suggests a need to “…be more sceptical about supposed innovations in QI methodologies and …seek more or better evidence before using them.”(98)

Buetow and Coster suggest that “…despite “pockets of improvement” there is little scientific evidence that CQI improves the quality of healthcare among large numbers of professionals or organisation-wide.”(136) In their survey based study into the effects of Quality Management (otherwise known as TQM) in Australian emergency medicine, Kennedy et al. found that views were mixed on whether QM has had a major impact of care delivery. While some respondents suggested that little of substance has arisen out or the ‘quality’ movement in health care, overall the principles and practices of quality management has established a sound foothold in emergency departments.(137)

In their 2006 study into the effectiveness of evidence based quality improvement (EBQI) (a modified version of CQI) on practice performance with patients with depression, Rubenstein et al. found that evidence based quality improvement had perceptible but modest effects. The study was undertaken by 2 different primary health care systems, collecting qualitative data on the design and implementation process. Previous studies testing the impact of CQI for depression had showed no effects.(138)

What hinders or helps?

Barriers

According to Solberg et al., organizations beginning to implement a CQI program should first consider whether it has chosen the right process to improve, and

identified any external or internal barriers to success.(42) There are a range of barriers to successful implementation of CQI initiatives. (30, 42, 101) It can be a long process (101, 139) and be resource intensive.(7, 139) Not all CQI initiatives succeed and occasionally when the barriers are significant or critical milestones are not met, the initiative may be scrapped out of frustration, lack of progress or because the organisation is too weakened to continue.(101) Says Berwick, “All improvement is change, but not all change is improvement.”(90) Undertaking QI can be a major decision for an organisation and its influence can have long lasting effects.(101)

Resistance to change itself (42, 101, 105) can be a significant barrier to successful implementation. Clinicians have been slow to understand and support CQI (7) and doctors’ behaviour is said to be “…notoriously hard to change.”(140) Potential changes in work structure and/or work mix for health professionals, as well as ingrained philosophies and mindsets may further discourage adoption.(101) In the hospital setting, structural barriers can also include that improvements are often focused at the departmental level rather than at an overall improvement to the system.(141)

Quality improvement is unlikely to the improve quality of care in a hospital setting unless the program is in line with organisational financial, strategic and market imperatives. Both the internal and external environments need to be considered to ensure that they are supporting effective CQI.(142) Factors that influence the implementation of CQI in larger hospitals are generally related to attitudes to the model itself and practical and personal obstacles.(143)

QI is more likely to be successfully implemented and therefore, positively affect quality of care; when hospitals place a high priority on successful implementation and properly resource the process. Conversely, hospitals that adopt a strategy of QI without sufficient resources may find they lose support for QI as day to day patient care needs compete for scarce financial and staff resources.(142)

In a study conducted by Geboers et al. into the attitudes of GPs and their practice staff to CQI, barriers most often mentioned included the amount of work and the tendency to postpone actions until external support was provided. Having doubts as to whether it was worth all the work was regarded most often as an important personal obstacle and difficulties in changing fixed routines was reported. Events which impact on the whole of practice such as rebuilding or computerization were seen as barriers, while physicians noted that “…interventions not directly related to practice work obstructed implementation of the model”.(143) According to one participant:

We had to invest too much time into it. Too much when compared with the results we achieved. Good ideas got stuck in logistic and financial obstacles. Quality management is good, but not feasible for our practice at the moment.(143)

Geboers et al argue that “…intensive support is necessary to implement and maintain continuous quality improvement in small scale practices.”(144) Practices are not always focused on implementing change where it is most needed; general practices encounter barriers to addressing organizational capacity; general practices require varying resources and support to make changes to improve their capacity for chronic disease management.(145) Further barriers to improvement

include poor finances, insufficient reimbursement, poor information technology and physicians that were too busy.(42, 143)

In discussing Quality Improvement Collaboratives, Mittman suggests that barriers to improvement are significant regardless and regardless of the size of the problem, the solutions required may be beyond the capacity of the QIC team to overcome.(119)

Enablers Theory and research suggest that successful quality improvement initiatives require a broad range of actions and supportive contextual factors.(119) For CQI to be successful in an organisation requires a shift from a culture that sees quality related to individual skills to one based on group performance and cooperation.(139) In a study that looked at whether training health care teams in CQI results in improved patient care and outcomes, Doran et al. found that team work is crucial to successful continuous quality improvement efforts. Front line health care professionals can positively affect the quality of care through the use of small group problem solving techniques and the application of a practical framework for structuring the teams improvement efforts.(115)

Teams that had participation from physicians were more likely to make changes in practice that led to improvements in outcomes for patients.(115) A physician proficient in collaboration would recognize that, to provide the best care for patients, physicians cannot work alone.(91)

However, a further study suggests that in the hospital setting, widespread physician participation in QI teams, while desirable, might not be necessary. Weiner et al. found that it was more important to obtain widespread participation of hospital staff and senior managers.(146) Solberg et al. suggest that significant quality improvement is dependant on changes to organisational systems and such changes require leadership and support from managers and decision makers within the organisation. (147)

Shortell et al. proposes that the autonomy of physicians mitigates against the creation of effective health care teams.(141) In any case says Enthoven, “…quick fixes will fail: changing cultures and processes will take time”. Indeed, it is argued that to implement CQI properly throughout an organization may take between five and seven years(7) or longer.(6)

Feifer claims that research conducted in large medical groups and smaller independent practices has found aspects of organisational culture or motivation to be important.(148) Glickman et al. however, suggest that there exists “…little empirical evidence that conclusively links organisational culture and performance”, and “There is a clear need to expand the evidence base to determine which cultural factors facilitate quality performance.”(123) Messner believes an organisation must firstly understand how amenable its culture is to change.(101)

A number of other factors are necessary for CQI success. These include the need for the intervention to be of major importance to the organisation (30); a readiness of the organisation for improvement(30, 42); capable leadership(30, 40, 42); appropriate information management/ technology systems(30, 123); trust with its

physicians; and a conductive external environment including regulatory, payment policy and competitive factors.(30)

Shortell et al. note the findings of a number of early studies which stress the importance of early physician involvement and support to successful quality improvement efforts. Evidence exists that CQI is likely to be more effective if physicians are meaningfully involved in the governance of the organization.(30) Regardless of their role and the manner in which they deliver care, clinicians may need to be convinced to make changes to conform to practice wide quality improvement efforts.(148)

A 1995 study found that the relationship between implementation and outcomes is largely about the size and culture of the hospital and implementing quality improvement in larger hospitals was a more difficult task. Further, having those with the technical capacity and authority to make the changes active in the planning of the program, appear to be most consistently linked with superior clinical improvement.(95)

Methods of quality improvement or assurance that focuses on systems is more likely to be more effective than those which seek to identify errors at the individual level for the purpose of apportioning blame.(9, 42, 90, 141, 148) In the medical profession, this equates to the need to shift from single interventions to change the behaviour of individual physicians to focus instead on the practice systems and organizations in which physicians work.(42) Says Batalden, “We will improve our current results only when we change the system that has produced them”.(91)

Successful implementation of CQI needs several other key factors: commitment of all staff(40, 143); commitment of upper management through direct involvement of CQI activities(69); training of management and front line staff in CQI methods and tools(69, 139); the development of org structures and procedures to support CQI initiatives(69) and to foster total employee involvement.(69, 148) Much of the success of quality improvement efforts will depend on clarifying roles and responsibilities and on the availability of data, appropriate incentives, and performance indicators.(149)

The success of any intervention in achieving sustainable QI will also depend on pre-existing organisational culture, climate, resources and priorities and must be introduced at the right time in national and local policymaking cycles.(150)

Physician ‘buy in’ can be difficult but is important. Organizations need to focus on those processes that concerns physicians either as suppliers or as key internal customers. Physicians deliver the ‘moments of truth’ that characterize the real output of service organizations.(69)

Swinehart argues that the quality of the service received by the external customer is linked to the efficiency and effectiveness of the internal customer supplier relationships. “The ‘internal supply chain’ must be clearly understood, carefully managed, and must play an important role in any successful CI effort”.(99)

A 2007 study found that a hospitals’ relative focus on improving systems and processes of care led to better quality indicators as the financial position of the hospital improved. According to a study by Alexander et al:

Results supported the proposition that QI implementation is unlikely to improve quality of care in hospital settings without a commensurate fit with the financial, strategic, and market imperatives faced by the hospital. Hospitals that are struggling financially will not be able to afford the sustained effort that is required to make QI programs successful. (142)

For CQI activities to be successful, Goldberg argues the effectiveness of implementing CQI programs that have already been proved successful.(151) Warden suggests that “Some believe that for quality improvement to be adopted across the industry, models must first be established by larger systems that have the infrastructure to implement a program in the clinical outcomes to demonstrate its success”.(152)

Chassin et al. are less hopeful, suggesting that even if there was “… a right set of strategies to encourage QI…” there is no ‘model’ organisation for others to emulate.(92) Kellet et al. state that “CQI appears to be a neat organisational concept rather than service reality, with CQI projects ubiquitous by their apparent absence”.(153)

What tools should we use to evaluate CQI? While CQI offer much in the way of potential benefits to medicine and health care more broadly, it is not easy to measure the success (or otherwise) of quality improvement.(7) The many factors that are brought to bear in a QI project do not always easily convert to easy to measure units(101), yet Random Controlled Trials, which rely on such standardization, are considered the ‘Gold Standard’ for testing medical interventions(101, 111, 119). Counte et al. suggest that a systematic method of understanding CQI and its effects would demand a randomised control trial, however, they concede that the diversity of QI programs works actively against the design and conduct of such a study.(7)

Berwick suggests that a number of changes to the approach to evidence in health care would accelerate the improvement of systems of care and practice. By itself, the usual OXO (Observe, Experiment, Observe again) experimental paradigm is not up to the task and adds that it is possible to rely on other methods without sacrificing rigor.

Many assessment techniques developed in engineering and used in quality improvement–statistical process control, time series analysis, simulations and factorial experiments–have more power to inform about mechanisms and contexts than do RCTs, as do ethnography, anthropology, and other qualitative methods. For these specific applications, these methods are not compromises in learning how to improve; they are superior.(154)

Greenfield cites an asthma project among school aged children which when evaluated through RCT showed no effects of a quality improvement intervention program. However, when a pre post study design was used to evaluate the same program, and despite acknowledged limitations of the study design, there were ‘compelling’ reasons to accept that the project improved the quality of care for children with asthma.(111)

Lindenauer similarly questions whether the methods used in traditional biomedical research are sufficient to evaluate QICs, and agrees that RCTs are not the best approach to evaluating quality improvement.(120) Says Pruessner;

(RCTs)…are excellent at establishing group norms and common causality, but they also fail absolutely in accounting for individual exceptions. Yet these individual exceptions are our patients, the people to whom we must apply this general knowledge.(155)

For some interventions and for a range of reasons, RCTs either cannot be performed or are of less value than other research designs.(111, 156, 157) Other criticisms include “…not all interventions can be tested in controlled trials”(158); “…and as currently conducted (they) are inefficient and have become more complex, time consuming(156)” and expensive.(111, 156)

While RCTs have their place in medical research(159), they have significant limitations.(159-161) Sturmberg suggests that long term studies using RCT are no different to any other prospective observational study. By focussing on one variable only, they exclude consideration of other factors other than the intended intervention as possible explanations for the observed outcomes.(159)

Brindis reports on how CQI, through an improvement in patient care processes, is associated with better clinical outcomes with emerging data demonstrating decreased mortality and asks whether RCTS are really necessary in this circumstance:

Is it necessary to have direct evidence that you have saved lives, or is it enough to show that you have improved processes that have a strong link to outcomes? (134)

“Where is the RCT?” is often the right question but at other times it is not.(154) For Berwick, a better question is:

‘What is everyone learning?’ Asking the question that way will help clinicians and researchers see further in navigating toward improvement.(154)

The paradigm of cause and effect vs complex adaptive systems

In addition to those mentioned earlier, numerous other studies have raised the issue of difficulty in attributing causality to interventions (16, 30, 74, 111, 162, 163) Specific reasons cited include the methodological challenges of measuring outcomes within complex and adapting organizations or health systems(74); patient and physician variables(162); concurrent events or a temporal trend.(41)

Is it sound to demand evidence of cause and effect when assessing the impact of continuous quality improvement programs? Neither illness nor human behaviour is predictable and so neither can be safely modelled in a simple cause and effect system.(164) In mechanical systems, the boundaries are easy to identify. For example, knowing what is or what is not a part of a motor car is not difficult. In human systems, the boundaries are less clear. People can belong to a number of different systems and their membership can change. This can complicate and lead to unpredictability.(140, 165)

Notwithstanding this knowledge, medical training and practice is largely underpinned by linear cause and effect mechanics(166-168) so the use of a cause and effect model to solve the problems within health is not surprising.(164) The human body can be thought of as a machine with illness attributed as a breakdown of its parts. “Such linear models drive us to break down clinical care into ever smaller divisions and to express with great accuracy and precision the intervention to be undertaken for each malfunction.”(164) Nature is rarely neither linear nor predictable. It is that non linearity of life that leads to unpredictability in complex systems.(168)

A complex adaptive system has been described as:

…a collection of individual agents with freedom to act in ways that are not always totally predictable, and whose actions are interconnected so that one agents’ action changes the context for other agents.(140)

Examples include the stock market, a termite colony or just about any collection of humans including a primary health care team.(140) Sturmberg uses the example of the flocking behaviour of birds to explain that:

…complex systems are self organising; pattern and organization develop iteratively through interactions among the systems components in the absence of any external supervisory influence.(167)

Given a complex system is not defined by its consistent components but rather its relationships or patterns of interaction, “… the behaviour of a complex adaptive system cannot be reduced to the behaviour of specific components and as such is said to be emergent”. They change, adapt and evolve.(167)

With health care more like the complex adaptive system than the well oiled machine of the earlier management paradigm, controlling the system to make or examine one intervention is not realistic (48). Complexity science suggests that an individuals’ state of health “…results from complex, dynamic, and unique interactions between different components of the overall system.”(164) Not all processes in health care/health organisations are linear nor are they easily articulated. For example, some aspects of quality may involve factors beyond the control of those providing direct care, but that may influence service delivery.(169, 170) Therefore causality is difficult if not impossible to attribute. Kazandjian suggests of the ‘fishbone’ diagram, a common CQI tool thus:

There is no causality analysis as the fish bone diagram is often misleadingly proposed to perform: the relationship between the bones and the head are probabilistic not deterministic. Causality could only be shown if the fish had one head and only one bone.(171)

Miles et al. held a similar view:

The relationship between process and outcome is probabilistic and not deterministic, and a poor clinical outcome can result from impeccable medical care and a good outcome may result despite poor care.(172)

According to Wilson and Goldschmidt, attribution of cause and effect to processes can only be assumed or inferred.

A patient’s recovery or death may have everything, nothing, or something to do with care processes. It is one thing to observe an effect and quite another to confidently attribute that effect to the preceding intervention.(6)

As in other complex adaptive systems, healthcare consists of highly adaptable elements (health professionals), inputs have non linear effects (small changes may create disproportionate effects) and the behaviour is emergent and sensitive to small changes and thus unpredictable over time.(140, 167, 168, 173, 174) Says, Grol et al. “…we need to find out which small changes, in which settings, can have such an impact”.(118)

Despite this unpredictability, it is possible to make generally true and practically useful statements about a complex system: there is an overall pattern. Plsek and Greenhalgh gives the example of a person presenting to their GP periodically until something is done to change this behaviour. While the exact timing cannot be predicted there is a reasonable assurance that the patient will present again unless they move out of the area and/or choose another GP. They add:

To cope with escalating complexity in healthcare we must abandon linear models, accept unpredictability, respect (and utilise) autonomy and creativity, and respond flexibly to emerging patterns and opportunities.(140)

Effective organisation and delivery of health care shouldn’t be focused on controlling processes or overcoming resistance to change.(175) The ‘complex real world’ is fuzzy, messy and unique and where problems and issues need to be understood within their context.(176) A complex adaptive systems view of organisations may allow a new way of thinking toward solving the challenges facing medical care.(167, 175) Such thinking would facilitate the establishment of goals and resources focussed on the whole system, “…rather than artificially allocating them to parts of the system.”(140)

A fresh look at QI research

Research into quality improvement cannot ignore important contextual factors (120, 154) and future research may be of more use if focused on the behaviours and actions of the participants themselves.(120) Healthcare issues are complex and based on the experiences of individuals. Complexity theory offers alternative perspectives on how predominately nonlinear health care systems operate.

In an area characterised by the often conflicting dictates of evidence, economics, equity and empowerment, the focus needs to be changed from academic research based paradigms to pragmatic health management approaches which reflect the context in which interventions are delivered. (177)

Rubenstein et al. note that ‘real world QI’ is complex and study designs need to account for such complexity in order to evaluate its effectiveness.(138) Similarly, in discussing quality improvement collaboratives, Mittman suggests the need for hybrid research approaches that “…differ significantly from the qualitative process-oriented reports that dominate the current QIC literature and from

quantitative, impact oriented studies” such as RCTs.(119) Likewise, Solberg et al. propose that consideration be given to range of research approaches which include “…case studies, observational and quasi-experimental designs, qualitative and mixed methods, practical clinical trials…”.(147)

Choosing the right interventions, evaluating their success and determining whether the results can be generalised presents a number of challenges according to Young et al. (122) They add:

Most QI interventions contain multiple strategies and it is difficult to determine which component is most effective.(122)

Similarly:

We have learnt that multifaceted strategies, combining different actions and measures linked to specific obstacles to change, are often successful, but do not have a good understanding of which components of such complex interventions are likely to be effective for different target groups.(118)

According to Grol and Grimshaw, more research is required into the important processes and elements of successful change. They argue that “…many different, sometimes competing approaches to changing practice…” exist, all claiming to be effective.(20) While there is limited research into their effectiveness or the conditions needed to implement quality programs successfully and what has been conducted often lacks rigor, some of what has been conducted can be still be used to inform decision making.(178) What is needed is a balance between feasibility and rigor.(179)

Why the lack of available evidence?

A number of commentators have expressed concern at such a lack of empirical knowledge as to how to bring about quality improvement (14, 103, 141, 180). Grimshaw et al. lament “…despite 30 years of research in this area, we still lack a robust, generalizable evidence base to inform decisions about QI strategies”.(181)

Further, despite the enthusiasm for embracing quality initiatives including CQI, it is difficult also to know which works best or indeed what success may look like. In 2004, Pope et al. wrote that even though quality improvement is a major goal of the healthcare systems of most developed nations and despite almost two decades of research, effective approaches to evaluation of quality improvement remain elusive.(182)

Although there are examples of particular small scale projects that have been shown to be effective, there is little evidence of that large scale benefits have brought important benefits or otherwise justified their cost. On the other hand:

…neither is there conclusive evidence that there are no benefits or that resources are being wasted. Such evidence may never exist: quality programmes are changing multicomponent interventions applied to complex organisations in a changing context with many short and long term outcomes, few of which can unambiguously be attributed to the intervention with the research designs that are possible.(183)

According to Booth et al. recent systematic reviews and meta-analysis provide a range of methods to potentially improve quality of health care and that while all methods have had success in some situations, none are effective in all situations.(16)

A number of commentators note that the published literature is limited in the reporting of effective implementation of CQI(7, 14, 16, 20, 92, 121, 122, 127, 184) and it appears that much of the research that has been undertaken occurred in hospitals. Unsystematic reviews of the evidence suggest that few healthcare organisations have successfully implemented a quality programme; however, the evidence provided by the studies is incomplete. Little is known about long term results or whether programmes have been sustained. Nonetheless, few other quality improvement programmes have been systematically studied or evaluated either.(183)

Publication bias

Stern and Simes claim that evidence for publication bias has been shown in many studies.(185) Wilson and Van De Weyden suggest that discussion and learning from improvement projects has been limited due to non publication of successful efforts.(89)

Others argue an alternative view; that publications “… are probably biased in favour of positive findings”.(119, 121) According to Mittman, this bias results from supply and demand factors. ‘Demand’ induced bias refers to the concentration of publications documenting successful quality improvement interventions in management and practitioner type journals where the readership is seeking answers to ‘how to?’ questions. Supply induced bias occurs when only successes are documented due to a focus by authors on quality improvement rather than research goals.(119) Stern and Simes argue that even evidence of treatment effectiveness from the ‘revered’ random controlled trial can be biased, particularly if trials reporting a positive effect of treatment are more likely to be published.(185)

Says Blumenthal and Kilo, CQI advocates argue that a lack of published literature supporting the effectiveness of CQI may be in part due to a lack of time or inclination of managers and those who oversee improvement activity to publish accounts of their results. Further, they cite an earlier Berwick paper and suggest:

…health professionals writing for medical journals may strip publications of references to CQI for fear of alienating sceptical physician reviewers, making it difficult to identify clinical advances that were fuelled by CQI activities. Whatever the reason, the lack of a robust scientific literature has clearly inhibited the acceptance of CQI methods among health professionals.(14)

Smith agrees that those working in quality improvement in health have a very poor record in publishing their articles, suggesting that this may be because they are too busy to publish or because journals won’t accept their submissions. Smith adds that it might be because improvement reports are hard to write and because the traditional IMRAD structure of scientific journals is unfriendly to such reports.(186)

Is it research or CQI?

The need to gain ethics approval in some circumstances can also be an issue contributing to a lack of published work in relation to CQI. Treating a CQI intervention in the same manner as a research project has the potential to seriously delay an intervention or cause it to be abandoned. Solberg et al. argue that:

A modified, sometimes retrospective IRB like review process must be available to assure that reasonable human subject protections were used while avoiding the current barriers to dissemination.(147)

Need to combine interventions

Grol et al. state that with the growing complexity of health care, simple improvement measures are rarely effective and it is not realistic to expect that one specific approach can solve all problems. “On the basis of current evidence none of the approaches to quality improvement can be regarded as superior; we might need them all to be succesful [sic] in achieving quality in health care”.(118)

Similarly, Oxman et al. state there are no ‘magic bullets’ for improving the quality of health care. However, there are a wide range of interventions available, that if used appropriately, could lead to important improvements in professional practice and patient outcomes.(43)

Despite mounting a spirited criticism of the enthusiastic adoption of CQI by the New Zealand Ministry of Health, Buetow and Coster nonetheless support the use of CQI as one of multiple approaches to quality improvement. They believe that CQI alone is not enough and other approaches cannot substitute for continuous quality improvement. “CQI is merely a tool, not the only one, and not necessarily the most important one, to help organisations, team and individuals improve in healthcare”.(136)

Says Chassin et al.: Whether one believes in regulation, continuous quality improvement, marketplace competition or payment incentives are the most effective way to improve the quality of care, evidence and experience to date suggests that none of these will prove to be the answer alone.(92)

Conclusions Early assumptions about the quality of health care were based on the premise that if the right infrastructure and education and training were provided, quality outcomes would result. These early concepts of quality were focused on competence with particular emphasis on the individual practitioner. Consequently, quality initiatives have largely been based on education and educational interventions.

For some time however, evidence has existed that such an approach can not guarantee quality. The least effective CME/CPD activities are most didactic or passive educational activities, while those interventions based on sound educational principles (needs analysis, alignment of educational methods with needs and evaluation of outcomes) have been associated with improved performance and health outcomes.

Quality assurance and accreditation have also been widely used across the health care sector and medical profession. Used alone however, these interventions do not guarantee quality or lead to continuous quality improvement. Indeed, most accreditation programs include CQI activities during accreditation cycles to ensure that organisations/practices do more than simply meet the minimum standards required.

Other interventions such as payment incentives, clinical audits and guidelines all have a contribution to make to quality improvement but again are least effective when used alone. The evidence shows that CQI is gaining widespread acceptance as a method of quality improvement across the health care industry, often in conjunction with other interventions.

While research into the effectiveness of continuous quality improvement has to date been limited, there is evidence that CQI interventions can lead to improvements within the health care system. Randomized trials testing the effectiveness of CQI as an approach to quality improvement in health have to date produced mixed results, however, a number of studies of CQI within hospital settings have identified improvements. There is an argument however, that most reported improvements relate to process and organisational improvements rather than demonstrated improvements to patient outcomes.

At the general practice level, though limited, some studies have shown small but statistically significant improvements in patient outcomes. As a quality improvement initiative, CQI may also be beneficial for general practice in that it is compatible with professional values in that it coherently combines the humanistic with the scientific values of the health professions. Additionally, CQI provides for simple but effective interventions that can involve the whole of the practice team. Further, quality interventions have the potential to improve patient care by focussing on the systems that support patient care, while reducing ineffective interventions. It should be noted however, that CQI interventions, practices and practitioners are not homogenous; one size does not fit all.

There are a number of conditions which are said to impact positively or negative on the successful implementation of CQI. These include a focus on improving the

system that supports the clinicians’ practice rather than a focus on the individual practitioner. Leadership from managers/decision makers is essential as is ‘buy in’ from staff at each level of the organisation or practice. Evidence has suggested that when practice teams engage in joint improvement partnerships and receive appropriate support, quality improves measurably. A further consideration is the choice of intervention. Organisations may choose interventions which may be costly and time intensive, only to find there has been little impact.

There is of course debate in regards to the lack of evidence for successful CQI, however, other quality improvement mechanisms have similar evidence doubts. In relation to CQI, perhaps the debate should be around ‘what is evidence?’ Study designs that have been used to evaluate CQI have failed to deliver the answers required with any certainty, particularly in relation to patient outcomes. There are also issues in seeking to attribute causality when health systems and the determinants of individual health are so complex, with so many confounding variables.

Finally, while a definitive answer to the question, “Does CQI improve patient outcomes?” remains elusive, their appears to be enough evidence to suggest that it can lead to process improvements, cost savings, improved practitioner satisfaction and staff empowerment among other benefits. Evidence is emerging that CQI is becoming the dominant quality improvement approach across health care– including the medical profession–and can lead to improved patient outcomes; particularly in conjunction with other interventions.

References 1. Moss F, Garside P. The importance of quality: sharing responsibility for improving patient care. BMJ. 1995;310(6985):996. 2. Donaldson LJ, Gray J. Clinical governance: a quality duty for health organisations. Qual Health Care. 1998;7(Suppl):S37-S44. 3. Wilson LL. Quality of care in hospitals: the steps consumers can take to avoid problems in healthcare. Australian Health Consumer. 2000 2000 Spring;3:6-8. 4. Sheldon T. Promoting health care quality: what role performance indicators? Qual Health Care. [Journal]. 1998;7 (Suppl):S45-S50. 5. Larson JS, Muller A. Managing the quality of health care. Journal of Health and Human Services Administration. 2002;25(3/4):261. 6. Wilson L, Goldschmidt P. Quality management in health care. Sydney: McGraw Hill; 1995. 7. Counte MA, Meurer S. Issues in the assessment of continuous quality improvement implementation in health care organizations. Int J Qual Health Care. 2001 June 1, 2001;13(3):197-207. 8. Crombie IK, Davies HTO. Beyond health outcomes: the advantages of measuring process. J Eval Clin Pract. 1998;4(1):31-8. 9. Schattner P, Markey P. Divisions, general practice and continuous quality improvement. What is the connection? . Aust Fam Physician. 2001;30(7):725-8. 10. Booth JL. The ACHS' changing approach to quality in health care. Australian Journal of Hospital Pharmacy. 1996 1996 Feb;26(1):86-9. 11. Balding C. From quality assurance to clinical governance. Aust Health Rev. 2008;32(3):383.

12. Smith R. The unending pursuit. Qual Saf Health Care. 1992;1(Suppl):45-7. 13. Nicholls S, Cullen, R. ONeill, S. Halligan, A. Clinical governance: its origins and its foundations. British Journal of Clinical Governance. 2000;5(3):172-8. 14. Blumenthal D, Kilo CM. A report card on continuous quality improvement. Milbank Q. 1998;76(4):625-48. 15. Pegram R, Wright D. Quality and general practice accreditation: the approach of Australian General Practice Accreditation Limited. Aust Health Rev. 1999 1999;22(4):161-71. 16. Booth BS, T; Harris, MF; Tomlins, R. Safety and quality in primary care: the view from general practice. Australian Journal of Primary Health. 2008;14(2):19-27. 17. Lanier D, C., Roland M, Burstin H, Knottnerus JA. Doctor performance and public accountability. Lancet. 2003;362(9393):1404. 18. Baker R, Roland M. General practice: continuous quality improvement since 1948. Br J Gen Pract. 2002;52:2-3. 19. Weeks WB, Robinson JL, Brooks WB, Batalden PB. Using early clinical experiences to integrate quality-improvement learning into medical education. Acad Med. 2000;75(1):81-4. 20. Grol R, Grimshaw. J. From best evidence to best practice: Effective implementation of change in patients' care. Lancet. 2003;362(9391):1225. 21. Miller SH. American board of medical specialties and repositioning for excellence in lifelong learning: maintenance of certification. J Contin Educ Health Prof. 2005;25(3):151-6. 22. Armson H, Kinzie S, Hawes D, Roder S, Wakefield J, Elmslie T. Translating learning into practice: Lessons from the practice-based small group learning program. Can Fam Physician. 2007 September 1, 2007;53(9):1477-85. 23. DHS. Working with general practice:Department of Human Services position statement. Melbourne, DHS; 2007 Contract No.: Document Number|. 24. Som CV. Clinical governance: a fresh look at its definition. Clinical Governance. 2004;9(2):87. 25. Scally G, Donaldson LJ. Looking forward: clinical governance and the drive for quality improvement in the new NHS in England. BMJ. 1998 July 4, 1998;317(7150):61-5. 26. Braine M, E. Clinical governance: applying theory to practice. Nurs Stand. 2006;20(20):56. 27. Kohn L, Corrigan J, Donaldson M, editors. To err is human: building a safer health system. Washington: National Academy Press; 2000. 28. IOM. Crossing the quality chasm: a new health system for the 21st century. Washington: National Academy Press; 2001. 29. Shine KI. Health care quality and how to achieve it. Acad Med. 2002;77(1):91-9. 30. Shortell S, M. , Bennett Charles L, Byck Gayle R. Assessing the impact of continuous quality improvement on clinical practice: What it will take to accelerate progress. Milbank Q. 1998;76(4):593. 31. Armstrong F. How safe are our hospitals? Australian Nursing Journal. 2004 2004 Apr;11(9):18-21. 32. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. MJA. 1995;163:458-71. 33. Van Der Weyden MB. The Bundaberg Hospital scandal: the need for reform in Queensland and beyond. MJA. [Editorial]. 2005;183(6):284-5.

34. Van Der Weyden MB. Australian healthcare reform: in need of political courage and champions. MJA. [Editorial]. 2003;179 (6):280-1. 35. Flynn J, Booth B, Portelli R. Professionalism and the quality framework. Aust Fam Physician. 2007;36(1-2):16-8. 36. Little P, Hayes S. Continuing professional development (CPD): GPs' perceptions of post-graduate education-approved (PGEA) meetings and personal professional development plans (PDPs). Fam Pract. 2003;20(2):192-8. 37. Campbell SM, McDonald R, Lester H. The experience of pay for performance in English family practice: a qualitative study. Ann Fam Med. 2008 May 1, 2008;6(3):228-34. 38. Teleki SS, Damberg CL, Pham C, Berry SH. Will Financial Incentives Stimulate Quality Improvement? Reactions From Frontline Physicians. Am J Med Qual. 2006 November 1, 2006;21(6):367-74. 39. Turner T, Misso M, Harris C, Green S. Development of evidence-based clinical practice guidelines (CPGs): comparing approaches. Implement Sci. 2008;3(1):45. 40. Wilcock P, Campion-Smith C. Never mind the quality: feel the improvement. Qual Health Care 1998;7 (4):181. 41. Solberg LI, Crain AL, Sperl-Hillen JM, Hroscikoski MC, Engebretson KI, O'Connor PJ. Effect of improved primary care access on quality of depression care. Ann Fam Med. 2006 January 1, 2006;4(1):69-74. 42. Solberg LI. Improving medical practice: a conceptual framework. Ann Fam Med. 2007 May 1, 2007;5(3):251-6. 43. Oxman AD, Thomson MA, Davis DA, Haynes B. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J. 1995;153:1423-31. 44. Davis D, O'Brien MAT, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999 September 1, 1999;282(9):867-74. 45. Stahl SM, Grady M, Santiago G, Davis RL. Optimizing outcomes in pychopharmacology continuing medical education (CME): measuring learning and attitudes that may predict knowledge translation into clinical practice. Focus. 2006 September 1, 2006;4(4):487-95. 46. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995;274(23). 47. Bellande BJ. The CME professional: challenges and opportunities in reforming CME. J Contin Educ Health Prof. 2005;25(3):203-9. 48. Booth B, Snowdon T. A quality framework for Australian general practice. Aust Fam Physician. 2007;36(1/2):8. 49. Sachdeva AK. The new paradigm of continuing education in surgery. Arch Surg. 2005 March 1, 2005;140(3):264-9. 50. Miller SH, Thompson JN, Mazmanian PE, Aparicio A, Davis DA, Spivey BE, et al. Continuing medical education, professional development, and requirements for medical licensure: A white paper of the Conjoint Committee on Continuing Medical Education. J Contin Educ Health Prof. 2008;28(2):95-8.

51. Moore DE, Jr., Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009 2009 Winter;29(1):1-15. 52. Davis D, Evans M, Jadad A, Perrier L, Rath D, Ryan D, et al. The case for knowledge translation: shortening the journey from evidence to effect. BMJ. 2003 July 3, 2003;327(7405):33-5. 53. Hakkennes S, Green S. Measures for assesing practice change in medical practitioners. Implement Sci. 2006;1(29). 54. Beyer M, Gerlach FM, Flies U, Grol R, Krol Z, Munck A, et al. The development of quality circles/peer review groups as a method of quality improvement in Europe.: Results of a survey in 26 European countries. Fam Pract. 2003 August 1, 2003;20(4):443-51. 55. Nahrwold DL. Continuing medical education reform for competency-based education and assessment. J Contin Educ Health Prof. 2005;25(3):168-73. 56. Asadoorian J, Batty HP. An evidence-based model of effective self-assessment for directing professional learning. J Dent Educ. 2005 December 1, 2005;69(12):1315-23. 57. Hulscher ME, van Drenth BB, van der Wouden JC, Mokkink HG, van Weel C, Grol RP. Changing preventive practice: a controlled trial on the effects of outreach visits to organise prevention of cardiovascular disease. Qual Saf Health Care. 1997 March 1, 1997;6(1):19-24. 58. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from flexner to competencies. Acad Med. 2002;77(5):361-7. 59. Veninga CCM, Ov PL, Wahlstrom R, Muskova M, Denig P, Berkhof J, et al. Evaluating an educational intervention to improve the treatment of asthma in four European countries. Am J Respir Crit Care Med. 1999 October 1, 1999;160(4):1254-62. 60. May F, Simpson D, Hart L, Rowett D, Perrier D. Experience with academic detailing services for quality improvement in primary care practice. Qual Saf Health Care. 2009 June 1, 2009;18(3):225-31. 61. Boom JA, Nelson CS, Laufman LE, Kohrt AE, Kozinetz CA. Improvement in provider immunization knowledge and behaviors following a peer education intervention. Clinical Pediatrics. 2007 October 1, 2007;46(8):706-17. 62. Van den Hombergh P, Grol R, van den Hoogen HJM, van den Bosch W. Practice visits as a tool in quality improvement: mutual visits and feedback by peers compared with visits and feedback by non-physician observers. Qual Health Care. 1999 September 1, 1999;8(3):161-6. 63. Lam-Antoniades M, Ratnapalan S, Tait G. Electronic continuing education in the health professions: an update on evidence from RCTs. J Contin Educ Health Prof. 2009 2009 Winter;29(1):44-51. 64. Lancaster J. Why 'MOPS' should become 'CPD' (continuing professional development). Australas Psychiatry. 2002;10(3):214-6. 65. Knowles MS, Holton III EF, Swanson RA. The adult learner: the definitive classic in adult education and human resource development [E-book]. Printed in the USA: Elsevier Butterworth Heinemann; 2005 [cited 2009 11th September]. Available from: http://reader.eblib.com.au.ezproxy-m.deakin.edu.au/Reader.aspx?p=232125&o=154&u=37Pt169pypnPvrY13gDm1w%3D%3D&t=1252647383&h=7009605bb43a7835497291f06cfd571eb7fd83a1&s=2383360&ut=484&pg=1&r=img#.

66. Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med. 1989 January 5, 1989;320(1):53-6. 67. Colton D. Quality improvement in health care: conceptual and historical foundations. Eval Health Prof. 2000 March 1, 2000;23(1):7-42. 68. RACGP. QA&CPD program handbook for general practitioners 2008-2010 triennium. South Melbourne: Royal Australian College of General Practitioners; 2007. 69. Berwick DM, Godfrey AB, Roessner J. Curing health care: new strategies for quality improvement. San Francisco Jossey-Bass; 1990. 70. Buetow SA, Roland M. Clinical governance: bridging the gap between managerial and clinical approaches to quality of care. Qual Health Care. 1998;8:184-90. 71. Wareham N. External monitoring of quality health care in the United States. Qual Health Care. 1994;3:97-101. 72. Nisselle P. Managing medical indemnity:must we choose between quality assurance and risk management? Med J Aust. 2004;181(2):64-5. 73. Buetow SA, Wellingham J. Accreditation of general practices: challenges and lessons. Qual Saf Health Care. 2003 April 1, 2003;12(2):129-35. 74. Al Tehewy M, Salem B, Habil I, El Okda S. Evaluation of accreditation program in non-governmental organizations' health units in Egypt: short-term outcomes. Int J Qual Health Care. 2009 June 1, 2009;21(3):183-9. 75. O'Leary D, S. . Accreditation's role in reducing medical errors. BMJ. 2000;320(7237):727. 76. Davies HTO. Performance management using health outcomes: in search of instrumentality. J Eval Clin Pract. 1998;4(4):359-62. 77. Johnston S, Dahrouge S, Hogg W. Gauging to gain: primary care performance measurement. Can Fam Physician. 2008 September 1, 2008;54(9):1215-7. 78. Werner RM, Asch DA. Clinical concerns about clinical performance measurement. Ann Fam Med. 2007 March 1, 2007;5(2):159-63. 79. Kapur N. On the pursuit of clinical excellence. Clinical Governance. 2009;14(1):24. 80. Frey JJ, III. In this iIssue: doctor-patient, doctor-system, doctor-public. Ann Fam Med. 2008 May 1, 2008;6(3):194-5. 81. McGovern MP, Boroujerdi MA, Taylor MW, Williams DJ, Hannaford PC, Lefevre KE, et al. The effect of the UK incentive-based contract on the management of patients with coronary heart disease in primary care. Fam Pract. 2008 February 1, 2008;25(1):33-9. 82. Gosfield AJ, Reinertsen JM. Finding common cause in quality: confronting the physician engagement challenge. Physician Exec 2008;34(2):26. 83. Galvin RS. The business case for quality. Health Aff. 2001;20(6):57. 84. O'Brien J, McCulloch I. Is there clinical governance after the new GP contract's Quality and Outcomes Framework? Clinical Governance Bulletin. 2005;5(6):11. 85. Peak M, Burke R, Ryan S, Wratten K, et al. VIEWPOINT: Clinical governance - the turn of continuous improvement? Clinical Governance. 2005;10(2):98. 86. Franks AJ. Clinical governance as a restructuring of quality assurance processes: Shifting the focus from corporate to clinical action. British Journal of Clinical Governance. 2001;6(4):259. 87. Swerissen HJ, L. Clinical governance in community health care settings: evidence and issues. Australian Journal of Primary Health. 2005;11(2):26-31.

88. Brown A, Mason V, Lyon A. Strengthening clinical governance in community health [Paper in special issue: Safety and Quality in Primary and Community Health Care. Renhard, Russell and Einfeld, Steve (eds).]. Australian Journal of Primary Health. 2008 Aug 2008;14(2):11-8. 89. Wilson RM, Van Der Weyden MB. The safety of Australian healthcare: 10 years after QAHCS. 2005. p. 260-1. 90. Berwick DM. Improvement, trust, and the healthcare workforce. Qual Saf Health Care. 2003 December 1, 2003;12(90001):i2-6. 91. Batalden P, Splaine M. What will it take to lead the continual improvement and innovation of health care in the twenty-first century? Qual Manag Health Care. 2002 2002 Fall;11(1):45-54. 92. Chassin MR, Galvin RW, and the National Roundtable on Health Care Q. The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998 September 16, 1998;280(11):1000-5. 93. Womack JP, Jones DT, Roos D. The machine that changed the world. New York: Simon and Schuster; 1990. 94. Colton D. Health care quality improvement: resources. Eval Health Prof. 2000 March 1, 2000;23(1):107-16. 95. Shortell SM, O Brien JL, Carman JM, Foster RW, et al. Assessing the impact of continuous quality improvement/total quality management: Concept versus implementation. Health Serv Res. 1995;30(2):377. 96. Weiner B, J., Shortell Stephen M, Alexander J, A. Promoting clinical involvement in the hospital quality improvement efforts: the effects of top management, board, and physician leadership. Health Serv Res. 1997; 32(4):491-510. 97. Mold JW, Peterson KA. Primary care practice-based research networks: working at the interface between research and quality improvement. Ann Fam Med. 2005 May 1, 2005;3(suppl_1):S12-20. 98. Walshe K. Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. Int J Qual Health Care. 2009 June 1, 2009;21(3):153-9. 99. Swinehart KD, Smith AE. Internal supply chain performance measurement: A health care continuous improvement implementation. Int J Health Care Qual Assur. 2005;18(6/7):533. 100. Schouten LMT, Hulscher MEJL, Everdingen JJEv, Huijsman R, Grol RPTM. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008 June 28, 2008;336(7659):1491-4. 101. Messner K. Barriers to implementing a quality improvement program. Nurs Manage. 1998;29(1):32-5. 102. Salman GF. Innovations in clincial practice. J Gen Intern Med 2005;20(9):862. 103. Ferris TG, Dougherty D, Blumenthal D, Perrin JM. A report card on quality improvement for children's health care. Pediatrics. 2001 January 1, 2001;107(1):143-55. 104. Barton AJPRN, Danek GPRN, Johns PMSNRN, Coons MMSNRN. Improving patient outcomes through CQI: vascular access planning. J Nurs Care Qual. 1998;13(2):77-85. 105. Ennis K, Harrington D. Quality management in Irish healthcare. Int J Health Care Qual Assur. 1999;12(6):232-43. 106. Harris S, Buchinski B, Gryzbowski S, Janssen P, Mitchell GWE, Farquharson D. Induction of labour: a continuous quality improvement and peer review program to improve the quality of care. CMAJ. 2000 October 31, 2000;163(9):1163-6.

107. Eagle KA, Montoye CK, Riba AL, DeFranco AC, Parrish R, Skorcz S, et al. Guideline-based standardized care is associated with substantially lower mortality in medicare patients with acute myocardial infarction: the American College of Cardiology's guidelines applied in practice (GAP) projects in Michigan. J Am Coll Cardiol. 2005 October 4, 2005;46(7):1242-8. 108. Maguerez G. Evaluation of 60 Continuous Quality Improvement Projects in French Hospitals. Int J Qual Health Care. 2001;13(2):89-97. 109. Cox S, Wilcock P, Young J. Improving the repeat prescribing process in a busy general practice. A study using continuous quality improvement methodology. Qual Health Care. 1999 June 1, 1999;8(2):119-25. 110. Fox P, Porter PG, Lob SH, Boer JH, Rocha DA, Adelson JW. Improving asthma-related health outcomes among low-income, multi-ethnic, school-aged children: results of a demonstration project that combined continuous quality improvement and community health worker strategies. Pediatrics. 2007;120(4):e902-11. 111. Greenfield S. Evaluating quality improvement. Pediatrics. 2007 November 1, 2007;120(5):1131-2. 112. Mohammadi SM, Mohammadi SF, Jerris RH, Morteza Z, Omid A. Introduction of a quality improvement program in a children's hospital in Tehran: design, implementation, evaluation and lessons learned. Int J Qual Health Care. 2007;19(4):237. 113. Jackson S, Bircher R. Transforming a run down general practice into a leading edge primary care organisation with the help of the EFQM excellence model. Int J Health Care Qual Assur. 2002;15(6/7):13. 114. Mitchell V. Same-day booking: Success in a Canadian family practice. Can Fam Physician. 2008 March 1, 2008;54(3):379-83. 115. Doran DMI, Baker GR, Murray M, Bohnen J, Zahn C, Sidani S, et al. Achieving clinical improvement: an interdisciplinary intervention. Health Care Manage Rev. 2002;27(4):42-56. 116. Verstappen WHJM, van Merode F, Grimshaw J, Dubois WI, Grol RPTM, van Der Weijden T. Comparing cost effects of two quality strategies to improve test ordering in primary care: a randomized trial. Int J Qual Health Care. 2004 October 1, 2004;16(5):391-8. 117. Ioannidis G, Papaioannou A, Thabane L, Gafni A, Hodsman A, Kvern B, et al. Canadian Quality Circle pilot project in osteoporosis: Rationale, methods, and feasibility. Can Fam Physician. 2007 October 1, 2007;53(10):1694-700. 118. Grol R, Baker GR, Moss F. Quality improvement research In: Grol R, Baker GR, Moss F, editors. Quality improvement research : the science of change in health care. London: BMJ Publishing; 2004. p. 1-5. 119. Mittman B, S. Creating the evidence base for quality improvement collaboratives. Ann Intern Med. 2004;140(11):897. 120. Lindenauer PK. Effects of quality improvement collaboratives. BMJ. 2008 June 28, 2008;336(7659):1448-9. 121. Landon B, E., Wilson I, B. , McInnes K, Landrum MB, et al. Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: the EQHIV study. Ann Intern Med. 2004;140(11):887. 122. Young PC, Glade GB, Stoddard GJ, Norlin C. Evaluation of a learning collaborative to improve the delivery of preventive services by pediatric practices. Pediatrics. 2006;117(5):1468-77.

123. Glickman S, Baggett K, Krubert C, Peterson E, Schulman K. Promoting quality: the health-care organization from a management perspective. Int J Qual Health Care. 2007;19(6):341. 124. Biuso C, Newton B. Supporting quality care in general practice: the collaboratives approach. Australian Journal of Primary Health. 2008;14(2):46-52. 125. Øvretveit J, Bate P, Cleary P, Cretin S, Gustafson D, McInnes K, et al. Quality collaboratives: lessons learned from research. Qual Saf Health Care. 2005;11(4):345-51. 126. Schonlau M, Mangione-Smith R, Chan KS, Keesey J, Rosen M, Louis TA, et al. Evaluation of a quality improvement collaborative in asthma care: does it improve processes and outcomes of Care? Ann Fam Med. 2005 May 1, 2005;3(3):200-8. 127. Solberg LI. If you've seen one quality improvement collaborative. Ann Fam Med. 2005 May 1, 2005;3(3):198-9. 128. Ferguson TB, Jr., Peterson ED, Coombs LP, Eiken MC, Carey ML, Grover FL, et al. Use of continuous quality improvement to increase use of process measures in patients undergoing coronary artery bypass graft surgery: a randomized controlled trial. JAMA. 2003 July 2, 2003;290(1):49-56. 129. Rogers S, Humphrey C, Irwin N, Lister S, et al. Designing trials of interventions to change professional practice in primary care: Lessons from an exploratory study of two change strategies. BMJ. 2000;320(7249):1580. 130. Simpkins J, Divine G, Wang M, Holmboe E, Pladevall M, Williams L. Improving asthma care through recertification: a cluster randomized trial. Arch Intern Med. 2008;168(16):2240-8. 131. Bailie RS, Si D, O'Donoghue L, Dowden M. Indigenous health: effective and sustainable health services through continuous quality improvement. Med J Aust. 2007;186(10):525-7. 132. Frijling BD, Lobo CM, Hulscher MEJL, Akkermans RP, van Drenth BB, Prins A, et al. Intensive support to improve clinical decision making in cardiovascular care: a randomised controlled trial in general practice. Qual Saf Health Care. 2003 June 1, 2003;12(3):181-7. 133. MCC. Using CQI to impact cancer early detection in the primary care setting. Michigan US: Michigan Cancer Consortium (MCC); 2005. 134. Brindis RG, Dehmer GJ. Continuous quality Improvement in the cardiac catheterization laboratory: are the benefits worth the cost and effort? Circulation. 2006 February 14, 2006;113(6):767-70. 135. Nutting PA, Dickinson LM, Rubenstein LV, Keeley RD, Smith JL, Elliott CE. Improving detection of suicidal ideation among depressed patients in primary care. Ann Fam Med. 2005 November 1, 2005;3(6):529-36. 136. Buetow S, Coster G. Is New Zealand according too much importance to continuous quality improvement in healthcare? Journal of the New Zealand Medical Association. 2004;117(1198). 137. Kennedy MR, Boyce NW, Logan ME. Quality management in Australian emergency medicine: translation of theory into practice. Int J Qual Health Care. 1999 August 1, 1999;11(4):329-36. 138. Rubenstein LV, Meredith LS, Parker LE, Gordon NP, Hickey SC, Oken C, et al. Impacts of evidence-based quality improvement on depression in primary care: a randomized experiment. J Gen Intern Med 2006;21(10):1027. 139. Enthoven AC, O'Reilly Denis St J, Byford S, Torgerson D, J. , Raftery J. Education and debate: A promising start, but fundamental reform is needed. BMJ. 2000;320(7245):1329.

140. Plsek P, Greenhalgh T. The challenge of complexity in health care. BMJ. 2001;323(7313):625-8. 141. Shortell SM, Singer SJ. Improving patient safety by taking systems seriously. JAMA. 2008 January 30, 2008;299(4):445-7. 142. Alexander J, A., Weiner B, J. , Shortell S, M., Baker L, C. Does quality improvement implementation affect hospital quality of care? Hosp Top. 2007;85(2):3. 143. Geboers H, Mokkink H, Van Montfort P, Van Den Hoogen H, Van Den Bosch W, Grol R. Continuous quality improvement in small general medical practices: the attitudes of general practitioners and other practice staff. Int J Qual Health Care. 2001 October 1, 2001;13(5):391-7. 144. Geboers H, van der Horst M, Mokkink H, van Montfort P, van den Bosch W, van den Hoogen H, et al. Setting up improvement projects in small scale primary care practices: feasibility of a model for continuous quality improvement. Qual Health Care. 1999 March 1, 1999;8(1):36-42. 145. Amoroso C, Proudfoot J, Bubner T, Swan E, Paola. E, Christopher. B, et al. Quality improvements activities associated with organisational capacity in general practice. Aust Fam Physician. 2007;36(1/2):81. 146. Weiner BJ, Alexander J, A. , Shortell S, M., Baker L, C. , Becker M, Geppert Jeffrey J. Quality improvement implementation and hospital performance on quality indicators. Health Serv Res. 2006;41(2):307-34. 147. Solberg LI, Elward KS, Phillips WR, Gill JM, Swanson G, Main DS, et al. How can primary care cross the quality chasm? Ann Fam Med. 2009 March 1, 2009;7(2):164-9. 148. Feifer C, Nemeth L, Nietert PJ, Wessell AM, Jenkins RG, Roylance L, et al. Different paths to high-quality care: three archetypes of top-performing practice sites. Ann Fam Med. 2007 May 1, 2007;5(3):233-41. 149. Koeck C. Time for organisational development in healthcare organisations. BMJ. 1998 November 7, 1998;317(7168):1267-8. 150. Macfarlane F, Greenhalgh T, Schofield T, Desombre T. RCGP Quality Team Development programme: an illuminative evaluation. Qual Saf Health Care. 2004 October 1, 2004;13(5):356-62. 151. Goldberg H, I. Building healthcare quality: if the future were easy, it would be here by now. Frontiers of Health Services Management. 1998;15( 1):40-2. 152. Warden G. Leadership in quality: some things to think about. Qual Manag Health Care. 2002 2002 Fall;11(1):1-2. 153. Kellett S, Clarke S, Matthews L. Continuous quality improvement using a practice improvement model. Clinical Governance. 2005;10(3):206. 154. Berwick DM. The science of improvement. JAMA. 2008;299(10):1182-4. 155. Pruessner HT, Hensel WA, Rasco TL. The scientific basis of generalist medicine Acad Med. 1992;67(4):232-5. 156. Luce BR, Kramer JM, Goodman SN, Connor JT, Tunis S, Whicher D, et al. Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change. Ann Intern Med. 2009 August 4, 2009;151(3):206-9. 157. Carey RG. Measuring health care quality: how do you know your care has improved? Eval Health Prof. 2000 March 1, 2000;23(1):43-57. 158. Harvey G, Wensing M. Methods for evaluation of small scale quality improvement projects. Qual Saf Health Care. 2003;12(3):210. 159. Sturmberg JP, Martin CM. The foundations of primary care: daring to be different. Oxford: Radcliffe; 2007.

160. Martin CB, Sturmberg JP. Perturbing ongoing conversations about systems and complexity in health services and systems. J Eval Clin Pract. 2009;15(3):549-52. 161. Herman J. Shortcomings of the randomized controlled trial: a view from the boondocks Herman, Joseph. Journal of Evaluation in clinical practice 4, 4 283-286 J Eval Clin Pract. 1998;4(4):283-6. 162. Hahn KA, Ferrante JM, Crosson JC, Hudson SV, Crabtree BF. Diabetes flow sheet use associated with guideline adherence. Ann Fam Med. 2008 May 1, 2008;6(3):235-8. 163. Solberg LI, Hroscikoski MC, Sperl-Hillen JM, Harper PG, Crabtree BF. Transforming medical care: case study of an exemplary, small medical group. Ann Fam Med. 2006 March 1, 2006;4(2):109-16. 164. Wilson T, Holt T, Greenhalgh T. Complexity science: complexity and clinical care. BMJ. 2001 September 22, 2001;323(7314):685-8. 165. Topolski S. Understanding health from a complex systems perspective. J Eval Clin Pract. 2009;15(3):749-54. 166. Smith R, O'Grady L, Jadad AR. In search of health. J Eval Clin Pract. [Editorial]. 2009;15(3):743-4. 167. Sturmberg JP, Martin CB. Complexity and health-yesterday's traditions, tomorrow's future. J Eval Clin Pract. 2009;15(3):543-8. 168. Katerndahl DA. Lessons from Jurassic Park: patients as complex adaptive systems. J Eval Clin Pract. 2009;15(3):755-60. 169. Colton D. The design of evaluations for continuous quality improvement. Eval Health Prof. 1997 September 1, 1997;20(3):265-85. 170. Gonella JS. Evaluation of competence, performance, and health care. Journal of Medical Education. 1979;54(10):825-7. 171. Kazandjian VE. The contribution of epidemiology to CQI: a commentary. Int J Qual Health Care. 1996;8(4):351-7. 172. Miles A, Bentley P, Polychronis A, Grey J, Price N. Recent progress in health services research: on the need for evidence-based debate. J Eval Clin Pract 1998;4(4):257-65. 173. Kelley M, A. , Tucci J, M. Bridging the quality chasm. BMJ. 2001;323(7304):61. 174. Leykum LK, Pugh J, Lawrence V, Parchman M, Noël P, Cornell J, et al. Organizational interventions employing principles of complexity science have improved outcomes for patients with Type II diabetes. Implement Sci. 2007;2(28). 175. Plsek PE, Wilson T. Complexity science: complexity, leadership, and management in healthcare organisations. BMJ. 2001 September 29, 2001;323(7315):746-9. 176. Fraser SW, Greenhalgh T. Coping with complexity:educating for capability. BMJ. 2001;324:709-803. 177. Kernick D, Stead J, Dixon M. Moving the research agenda to where it matters. BMJ. 1999;319(7204):206. 178. Øvretveit J, Gustafson D. Using research to inform quality programmes. BMJ. 2003;326(7392):759. 179. Needham DM, Sinopoli DJ, Dinglas VD, Berenholtz SM, Korupolu R, Watson SR, et al. Improving data quality control in quality improvement projects. Int J Qual Health Care. 2009 April 1, 2009;21(2):145-50. 180. Brouwer HJ, Bindels PJE, Weert HC. Data quality improvement in general practice. Fam Pract. 2006;23(5):529.

181. Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, et al. Toward evidence-based qality improvement. J Gen Intern Med 2006;21(S2):S14-S20. 182. Pope C, Van Royen P, Baker R. Qualitative methods in research on healthcare quality. In: Grol R, Baker GR, Moss F, editors. Quality improvement research : the science of change in health care. London: BMJ Publishing; 2004. p. 51-63. 183. Ovretveit J, Gustafson D. Using research to inform quality programmes. BMJ. 2003;326(7392):759. 184. Jones K, Piterman L. Promoting best practice in general practitioner management of osteoarthritis of the hip and knee: arthritis and musculoskeletal quality improvement (AMQuIP) program. Australian Journal of Primary Health. 2007;13(2):104-12. 185. Stern JM, Simes JR. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ. 1997;315(7109):640. 186. Smith R. Quality improvement reports: a new kind of article. BMJ. 2000 December 9, 2000;321(7274):1428.