Embed Size (px)
Citation preview
Title: A Systematic review of quality measures used in Primary Care DentistryM. J. Byrne1, M. Tickle1, A-M. Glenny1, S. Campbell2, T. Goodwin1 L. O’Malley1
1. Division of Dentistry, School of Medical Sciences, Faculty of Biology, Medicine and Health,
University of Manchester
2. Director NIHR Patient Safety Translational Research Centre, Division of Population Health,
HSR & Primary Care, School of Health Sciences, Faculty of Biology, Medicine and Health,
University of Manchester
Key words: Quality, measurement, improvement, indicators
Correspondence to:
Matthew Byrne
NIHR Academic Clinical Fellow in Primary Care Dentistry,
Division of Dentistry,
School of Medical Sciences,
Faculty of Biology, Medicine and Health,
University of Manchester,
Oxford Road,
M13 9PL
Abstract word count: 246
Main Text word count: 2999
1
Abstract
Introduction ‘Quality’ in primary care dentistry is poorly defined. There are significant
international efforts focused on developing quality measures within dentistry. The aim of
this research was to identify measures used to assess quality in primary care dentistry and
categorise them according to which dimensions of quality they attempt to measure.
Methods Quality measures were identified from the peer-reviewed and grey literature.
Peer-reviewed papers describing the development and validation of measures were
identified using a structured literature search. Measures from the grey literature were
identified using structured searches and direct contact with dental providers and
institutions. Quality measures were categorised by domains of structure, process and
outcome and disaggregated dimensions of quality. Results 11 validated measure sets from
22 studies were identified from the peer-reviewed literature, comprising 9 patient
satisfaction surveys and 2 practice assessment instruments. From the grey literature, 24
measure sets, comprising 357 individual measures, were identified. Of these, 96 addressed
structure, 174 addressed process and 87 addressed outcome. Only 3 of these 24 measure
sets demonstrated evidence of validity testing. The identified measures failed to address
dimensions of quality such as efficiency and equity. Conclusions There has been a
proliferation in the development of dental quality measures in recent years. However, this
development has not been guided by a clear understanding of the meaning of quality. Few
existing measures have undergone rigorous validity or reliability testing. A consensus is
needed for a definition and the dimensions of quality in dentistry to develop core measures
to assess it.
2
IntroductionImproving quality in primary dental care is a goal of international interest (1, 2). To improve
the quality of healthcare, a definition of quality and the criteria used to assess quality must
be established(3). Most definitions of quality in primary care relate to medicine, for which
several have been described (4-13). Dentistry has a number of significant differences to
medicine, which has led to suggestion of the need for a specific definition of quality in
dentistry to be developed(3). Whilst some specific definitions for quality in dentistry have
been offered (14, 15), there is no agreed definition or conceptual framework available for
what quality means for primary dental care (1, 3, 16). The Wold Dental Federation (FDI) has
defined quality as ‘an iterative process involving dental professionals, patients and other
stakeholders to develop and maintain goals and measures to achieve optimal health
outcomes’(2). This statement highlights the importance of measurement of quality in the
process of improving patient care. However, the constituent parts of this and other
definitions need to be considered further to ensure that all of the key dimensions of quality
are captured. Furthermore, the view that dentistry is so different from other areas of
healthcare that it merits its own definition must be tested.
Many quality assurance and improvement schemes have been attempted but are hampered
by a weak evidence base(17). The Institute of Medicine (IoM) in the United States have
stated that quality measures in dentistry ‘lag far behind’ those in medicine and other health
professions(18). The IoM suggest that construction of quality measures would help to
improve oral health and reduce inequalities(18). A first step in this process is to develop a
comprehensive understanding of what type of measures have been developed and what
dimensions of quality they are attempting to measure. Quality is complex and multi-
dimensional; therefore, a structured approach to terminology usage is necessary to avoid
confusion. We propose the following terminology can be used to categorise quality
measures:
DomainsThe Donabedian systems-based model of quality states that quality measures may address
one or more of three domains: the structures that contribute to the delivery of care; the
3
processes of care; and the outcomes of care(6). Measures of structure include assessments
of the provision of facilities, staff and training. Process measures assess what the
practitioner actually does whereas outcomes measurements assess the impact of an
intervention(6). Evidence is required to show that the measurement and improvement of
processes will lead to an improvement in outcomes(19), for example, if placement of fissure
sealants (process) leads to prevention of caries (outcome).
DimensionsQuality may be disaggregated into different dimensions. These dimensions each give a
partial view of quality(7). The IoM definition of quality identifies dimensions of safety,
effectiveness, timeliness, patient-centeredness, efficiency and equity(13). There is no
consensus on the dimensions of quality that are most pertinent to dentistry.
An ideal measure set in dentistry would address each of the dimensions quality most
relevant to dentistry across the domains of structure, process and outcome.
The aim of this research was to identify measures used to assess quality in primary care
dentistry. Measures were assessed for their validity and reliability and were categorised
according to the IoM’s dimensions of quality and Donabedian’s domains. The identification
of these measures will assist in the production a framework to support further development
of quality measures in dentistry.
4
Methods Peer-reviewed LiteratureA systematic search strategy was used to identify measures from the published literature
(Appendix 1). Each identified measure was assessed for internal and external validity. The
constituent parts of each measure were assessed for the Donabedian domain and IoM
disaggregated dimension of quality of quality they measure.
Inclusion Criteria
Types of studies: Any cross-sectional or longitudinal study concerning the development or
validation of a quality measure in primary care dentistry, or that uses a previously reported
measure.
Types of measures: Measures that may be used by a dentist, patient or other stakeholder to
assess quality in a primary care setting.
Studies were only selected if they presented the process by which the measure was
developed and validated in sufficient detail. Measures whose validity was confirmed by
later studies were included.
Exclusion Criteria
Non peer-reviewed studies or opinion pieces, studies published in a language other than
English, studies published pre-1970 and studies that did not describe development or
validity testing of the measure were excluded
Measures of Oral Health Related Quality of Life (OHRQoL), being primarily epidemiological
tools, were excluded.
A structured systematic literature search was produced for use in MEDLINE (Appendix 1)
and adjusted for other databases. The Databases used were MEDLINE via OVID (1946 to
Present); Psychinfo via OVID (1806 to Present) EMBASE via OVID (1980 to Present), Health
and Psychosocial Instruments via OVID (1985 to Present) and Social Policy and Practice via
OVID.
5
The screening process was managed by using an Endnote Library; references were initially
screened by title and abstract by MB, AG, MT, SC and LO to remove irrelevant papers. The
remaining papers were double screened by MB and AG, with disagreements discussed with
MT. Full text papers were obtained for the remaining included studies. Reference lists of
these studies were hand searched to obtain key sources that described the development or
validation of the measures. Key journals were directly searched to account for articles that
may have been missing from the literature search strategy.
Grey LiteratureSearching of the grey literature was completed by using the OpenGrey database(20), and
through hand searching of the websites of large dental providers, dental associations,
insurers and government bodies in the English-speaking Nations (USA, Canada, UK,
Australia, New Zealand). Dental insurers and large corporate dental providers were
contacted directly for any quality measures they use. The National Quality Forum’s
‘Environmental Scan, Gap Analysis & Measure Topics Prioritization’ was consulted, as this
project had similar goals of measure identification (17). A number of the measures identified
within this study are no longer in use and were thus omitted. As measures from the grey
literature are actively measuring and affecting clinical practice, exclusion criteria were
limited to non-English references and publication prior to 1970
Evaluation and Categorisation of MeasuresAs there is no established measure to assess the quality of quality measures within primary
care dentistry, the measures were individually assessed for their internal and external
validity. Measures that did not demonstrate face validity were excluded. Evaluation of the
internal consistency of measures, such as Cronbach’s Alpha scores, were extracted. An
α<0.5 suggests unacceptable internal consistency of the measure(21). Measures of test-
retest reliability such as Intraclass correlation Coefficients and Spearman’s Rank Correlation
Coefficients were also extracted. Each measure was categorised according to the
Donabedian’s domains of Structure, Process and Outcome and then assessed against the six
disaggregated dimensions of quality of the IoM definition of quality.
6
Method of analysis and synthesisStructured tables were used to describe the data from each measure narratively to give an
overview of the domains of quality evaluated by each measure and the dimensions of
quality they address.
7
ResultsMeasures from the Peer-reviewed Literature
A flow diagram of the screening process is presented in Figure 1. A total of 543 papers were
generated from the structured literature search. 7 papers were found from further
searches. After removal of 49 duplicates, 501 papers were present. 285 were initially
excluded due to irrelevance. Full text papers of the remaining 216 papers were assessed. In
total, 22 papers met the inclusion criteria. Within these, 11 individual measure sets were
identified; 9 of these were patient satisfaction scales and 2 were practice assessment
instruments for use by a dentist or practice manager
Table 1 summarises the contents of each measure set and describes the internal validity and
reliability testing of the included measures, alongside further references that have used
these measurement tools. The 9 patient satisfaction scales use Likert-style scores to assess
satisfaction. These measures were to be completed by the patient receiving treatment, or
their parent/guardian. Similar ordinal rating scales were used by the measures described in
the two practice assessment tools. These measures were designed to be completed by a
dentist or manager in the dental practice.
All the included measure sets showed face validity. Cronbach’s Alpha was used to describe
internal validity in all of the measures. All were above the acceptable level of α=0.7(21),
except the DMS-BR(22) α=0.632 (questionable internal consistency) and components of the
DSQ(38). The validity of the DSQ is demonstrated by Chapko (44), the measurement concept
of ‘Cost’ was suggested as unreliable (α=0.47). All other measures in this set had α>0.6 .
Test-retest reliability was reported in 5 of the measures, using either intraclass correlation
coefficients or Spearman rank correlation coefficients, all of which showed acceptable
values.
The patient satisfaction scales broadly considered patient satisfaction with the care
received. For these measures, satisfaction can be considered an outcome of care. Where
specific questions address the procedural aspects of care, the domain of process is also
8
measured. The BiPD-Q(27) and BiOS-Q(29) measures were specifically related to satisfaction
with processes of care and the procedural elements of prosthetic dentistry and oral surgery.
Structure was measured in the practice self-assessment tools (DMS-BR(22), SOADC(23)) and
concerned the training of staff and provision of safety equipment and data management.
The questionnaire developed by Bahadori et al. (26) asked patients to rate the importance
of a number of structures and processes within a dental clinic, including the state of facilities
and the communication skills exhibited by the dentist.
The practice assessment measures (DMS-BR(22), SOADC(23)) assessed the dimensions of a.)
safety, with measures addressing the use and provision of Personal protective equipment
and b.) efficacy, using measures of ability of practice members to work as a team. The
patient satisfaction surveys covered a range of dimensions, for example: Safety - satisfaction
with cleanliness of facilities(30), effectiveness – satisfaction with treatment received(33),
patient-centeredness – perception of dentist caring about patient(32), timeliness –
satisfaction with waiting times to see a dentist(31), efficiency – patient satisfaction of cost
(38) and equity – patient perception of dentist acceptance of them as a person(33).
Measures from the Grey LiteratureIn total 24 collections of quality measures sets were identified, with a total of 357 individual
quality measures contained therein. Table 2 describes the measurement sets qualitatively in
terms of what attributes they attempt to measure, evidence of validity testing and
categorization of measures by domain and IoM dimensions. The majority of measures
within these sets (n=196/357) followed a numerator/denominator format, wherein the
patients receiving a treatment process or reporting an outcome were classed as the
numerator, and a target population was identified as the denominator. These types of
quality measure are presented as a percentage. A further 36 measures were patient
satisfaction measures using Likert-style ordinal rating scores. The Denplan Excel Quality
measures(68) contains 122 checklist style yes/no responses. The only measures that
described validity or test-retest reliability were those developed by the Dental Quality
Alliance(49-51). These measures have been developed according to the National Quality
Forum measure development guidance(69).
9
The majority (48.7%; 174/357) of measures assessed processes of care provision. Common
themes for process measures were the provision of fluoride, fissure sealants and annual
reviews. There was a high degree of repetition of these concepts across a number of
measure sets. A total of 24.4% (87/357) measured outcomes with patient satisfaction
ratings making up 52.9% (46/87) of these outcome measures. The remaining outcome
measures included measures of longevity of restorations, rates of complications and new
disease presence at recall. 26.8% (96/357) of the measures assessed structure, 84 of these
96 (87.5%) measures were from the Denplan Excel quality measures, which quantified
provision of equipment and staff within practices. The measures predominantly assessed
dimensions of effectiveness, patient centeredness, safety and timeliness of treatment.
There were few measures of efficiency or equity.
10
Quality Measures Gap analysis
The common themes identified in both the peer reviewed and grey literature measures are
compiled in Table 3. This table identifies the broad constituent element of quality the
measure attempts to capture (IoM Dimensions) and the nature of the measure
(Donabedian’s Domains). The categorisation of domain refers to the whether the measure
assesses the structures of dental care delivery, the processes that are undertaken or
outcomes that result from the delivery of care. As such, a measure of the provision of a
treatment or preventive program to a population is categorised as a process measure. This
analysis shows a proliferation of measures developed for the assessment of process and
outcomes within the dimensions of effectiveness and patient centeredness. Outcome
measurement is predominantly achieved via patient satisfaction measures. Significant gaps
in measures across the domains and dimensions of quality are evident.
11
Discussion
This systematic review describes 11 measure collections (167 total measures) from the peer-
reviewed literature and 24 measure collections (357 total measures) from the grey literature
that may be used to assess quality in primary care dentistry. This study is the first known
review of quality measures in primary care dentistry that uses a systematic review design,
with the use of a priori inclusion and exclusion criteria.
Despite the structured searching methodology, it is difficult to ensure that a comprehensive
list of measures has been captured, particularly those which appear in the grey literature.
The pace at which new measures are being developed outside of the peer reviewed
literature means further measures are likely to have been produced since searching was
completed. Whilst this search may not include every measure that is available, it does
display the major trends of how quality is currently measured and therefore viewed by the
dental profession. Only English language sources were used, as the cost associated with the
translation of foreign language measures and papers would be prohibitive. This provides a
view of quality measurement dominated by developed countries. Quality measures will,
necessarily, reflect the context and the priorities in which care is delivered; measures
formulated in the developed world may not be as relevant in less developed communities.
FDI called for an international consensus on understanding quality in dentistry (2) and the
influence of local context and priorities should not be underestimated in working towards
this goal.
Using valid and reliable tools to measure quality is vital in order to support day to day
quality assessment and improvement of dental care. The 11 measure sets from the peer
reviewed literature (22-24, 26, 27, 29, 30, 32, 33, 38) and 3 measure sets from the grey
literature (50, 56, 57) represent measures available to researchers to assess dimensions of
quality in relation to primary care dentistry. The majority of these measures showed
acceptable levels of internal validity, however their usefulness in delivering a clear picture of
quality and to support quality improvement is unclear.
12
The gap analysis categorised the measures by the IoM dimensions of quality and identified
important areas of care in which specific dental measures were absent. Some of these gaps
could possibly be populated by measures used successfully in other areas of healthcare. It
could also be the case that the IOM dimensions fail to adequately describe significant
elements of quality that are important to dentistry such as cosmetic care, functional
improvement and the discomfort and anxiety associated with dental procedures. Further
dimensions described in the literature include: tangibility(26, 70), responsiveness(26, 70),
empathy(26, 70), accessibility(7, 12), coordination and continuity of care(12),
comprehensiveness(12), technical quality(5), acceptability(71), legitimacy(71),
optimality(71), relevance(72), appropriateness(11) and ‘caring function’(11).
The majority of the measures within the grey literature focussed on processes of care. Core
themes of fluoride prescription, provision of fissure sealants and dental attendance were
identified from this search. However, without corresponding measures of resulting
outcomes, these process measures may have limited utility(19). The majority of outcome
measures identified in this search assessed patient satisfaction. However, a review of
patient satisfaction measures suggests that these are highly affected by disconfirmation and
attribution bias and are inherently unreliable(73). It has been suggested that the patient’s
perception of technical competence is based upon the communication and caring nature of
the dentist rather than the work they actually carry out (33). As the primary stakeholders of
primary care dentistry, it is intuitively important that patients are provided with a service
that provides satisfaction, however measures of patient satisfaction give only a limited
indication of overall quality of care provided.
Consideration also needs to be given to the reasons for collecting data on quality, the
various stakeholders in primary dental care have different priorities for their use. For
example, policy makers may want to use such data to improve equity and access for a
defined population. Dentists may wish to use quality measurement to fulfil a personal and
professional desire to improve the care for their patients or as a way of marketing
themselves or their practice. Patients may wish to use measures to compare the quality of
13
care provided by different dentists. Providers of dental care may wish to use quality data for
performance management of their dentists, or as a way of remunerating and incentivising
their dentists. Linking quality improvement to remuneration runs the risk of inducing
unintended behaviour change leading to alterations in the provision of care (74); this may
include services de-registering high-risk patients that threaten to lower their quality rating.
The majority of measures being used within primary care dentistry are being developed by
private companies, such as insurers or corporate care providers and are not being published
in peer reviewed journals. Two issues arise from this: firstly, time and resources are being
used on the development of measures that have already been developed elsewhere and
secondly, measures are being used without the scrutiny of the wider scientific community.
The authors believe that international collaboration is required to develop valid and reliable
measures that can be applied across a range of contexts.
14
Conclusions
This research has highlighted the lack of valid, reliable measures used to assess quality.
Current efforts in producing quality measures are being conducted with little academic
grounding leading to measures of limited validity and utility that fail to capture some key
elements of quality. International collaboration is required to agree a definition of quality
and a shared understanding of what quality means for dentistry. Consensus would guide the
production of valid, reliable measures within an agreed framework for the benefit of
patients.
Acknowledgements and Competing interestsNo further acknowledgements are made. The authors have no competing interests to
declare. No financial support was provided for this project.
15
References:
1. European Regional Organisation of the Fédération Dentaire Internationale. ERO-FDI WG Quality in Dentistry Report 2007-2010. 2010 [cited 03/04/2018] [Available from: www.erodental.org/ddc/ddid97.]2. FDI World Dental Federation. Quality in Dentistry. 2017 [cited 03/04/2018] [Available from: http://www.fdiworlddental.org/resources/policy-statements-and-resolutions/quality-in-dentistry.]3. Campbell S, Tickle M. What is quality primary dental care? Br Dent J. 2013;215(3):135-9.4. Maxwell RJ. Quality assessment in health. Br Med J (Clin Res Ed). 1984;288(6428):1470-2.5. Blumenthal D. Part 1: Quality of care--what is it? N Engl J Med. 1996;335(12):891-4.6. Donabedian A. The definition of quality and approaches to its assesment: Health Administration Press; 1980.7. Campbell SM, Roland MO, Buetow SA. Defining quality of care. Soc Sci Med. 2000;51(11):1611-25.8. Heath I, Rubinstein A, Stange KC, van Driel ML. Quality in primary health care: a multidimensional approach to complexity. BMJ. 2009;338:b1242.9. Soni Raleigh V, Foot C. Getting the measure of quality: opportunities and challenges. London: King's Fund; 2010.10. Royal College of General Practitioners. What sort of doctor? J R Coll Gen Pract. 1981;31(232):698-702.11. Caper P. Defining quality in medical care. Health Aff. 1988;7(1):49-61.12. Health Services Research Group. Quality of care: 1. What is quality and how can it be measured? CMAJ: Canadian Medical Association Journal. 1992;146(12):2153-8.13. Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press 2001.14. Tickle M, L OM, Brocklehurst P, Glenny AM, Walsh T, Campbell S. A national survey of the public's views on quality in dental care. Br Dent J. 2015;219(3):E1.15. Poorterman JH, Van Weert CM, Eijkman MA. Quality assurance in dentistry: the Dutch approach. International journal for quality in health care : Journal of the International Society for Quality in Health Care. 1998;10(4):345-50.16. Mills I, Batchelor P. Quality indicators: the rationale behind their use in NHS dentistry. Br Dent J. 2011;211(1):11-5.17. National Quality Forum. Oral Health Performance Measurement: Environmental Scan, Gap Analysis & Measure Topics Prioritization. 2012.18. Institute of Medicine and National Research Council Committee on Oral Health Access to Services. Improving Access to Oral Health Care for Vulnerable and Underserved Populations. Washington, DC: National Academies Press; 2011.19. Bader JD. Challenges in quality assessment of dental care. J Am Dent Assoc. 2009;140(12):1456-64.20. OpenGrey [Internet]. [cited 29.03.18]. Available from: http://www.opengrey.eu.
16
21. DeVellis R. Scale Development: theory and applications. Thousand Oaks, California: Sage; 2003.22. Gonzales PS, Martins IEF, Biazevic MG, Silva PR, Michel-Crosato E. Dental Management Survey Brazil (DMS-BR): creation and validation of a management instrument. Braz Oral Res. 2017;31:e26.23. Goetz K, Hasse P, Szecsenyi J, Campbell SM. Questionnaire for measuring organisational attributes in dental-care practices: psychometric properties and test-retest reliability. Int Dent J. 2016;66(2):93-8.24. Cheng BS, McGrath C, Bridges SM, Yiu CK. Development and evaluation of a Dental Patient Feedback on Consultation skills (DPFC) measure to enhance communication. Community Dent Health. 2015;32(4):226-30.25. Wong HM, Bridges SM, McGrath CP, Yiu CK, Zayts OA, Au TK. Impact of Prominent Themes in Clinician-Patient Conversations on Caregiver's Perceived Quality of Communication with Paediatric Dental Visits. PLoS One. 2017;12(1):e0169059.26. Bahadori M, Raadabadi M, Ravangard R, Baldacchino D. Factors affecting dental service quality. Int J Health Care Qual Assur. 2015;28(7):678-89.27. Reissmann DR, Hacker T, Farhan D, Heydecke G. The Burdens in Prosthetic Dentistry Questionnaire (BiPD-Q): development and validation of a patient-based measure for process-related quality of care in prosthetic dentistry. Int J Prosthodont. 2013;26(3):250-9.28. Hacker T, Heydecke G, Reissmann DR. Impact of procedures during prosthodontic treatment on patients' perceived burdens. J Dent. 2015;43(1):51-7.29. Reissmann DR, Semmusch J, Farhan D, Smeets R, Heiland M, Heydecke G. Development and validation of the Burdens in Oral Surgery Questionnaire (BiOS-Q). J Oral Rehabil. 2013;40(10):780-7.30. Kikwilu EN, Kahabuka FK, Masalu JR, Senkoro A. Satisfaction with urgent oral care among adult Tanzanians. Journal of Oral Science. 2009;51(1):47-54.31. Keller S, Martin CGC, Evensen CT, Mitton RH. The development and testing of a survey instrument for benchmarking dental plan performance Using insured patients' experiences as a gauge of dental care quality. Journal of the American Dental Association. 2009;140(2):229-37.32. Larsson BW, Bergstrom K. Adolescents' perception of the quality of orthodontic treatment. Scand J Caring Sci. 2005;19(2):95-101.33. Corah NL, O'Shea RM, Pace LF, Seyrek SK. Development of a patient measure of satisfaction with the dentist: the Dental Visit Satisfaction Scale. J Behav Med. 1984;7(4):367-73.34. Olausson M, Esfahani N, Ostlin J, Hagglin C. Native-born versus foreign-born patients' perception of communication and care in Swedish dental service. Swed Dent J. 2016;40(1):91-100.35. Sun N, Burnside G, Harris R. Patient satisfaction with care by dental therapists. Br Dent J. 2010;208(5):E9; discussion 212-3.36. Hakeberg M, Heidari E, Norinder M, Berggren U. A Swedish version of the Dental Visit Satisfaction Scale. Acta Odontol Scand. 2000;58(1):19-24.37. Stouthard ME, Hartman CA, Hoogstraten J. Development of a Dutch version of the Dental Visit Satisfaction Scale. Community Dent Oral Epidemiol. 1992;20(6):351-3.38. Davies AR, Ware JE, Jr. Measuring patient satisfaction with dental care. Soc Sci Med A. 1981;15(6):751-60.
17
39. Lee CT, Zhang S, Leung YY, Li SK, Tsang CC, Chu CH. Patients' satisfaction and prevalence of complications on surgical extraction of third molar. Patient Prefer Adherence. 2015;9:257-63.40. Milgrom P, Spiekerman C, Grembowski D. Dissatisfaction with dental care among mothers of Medicaid-enrolled children. Community Dent Oral Epidemiol. 2008;36(5):451-8.41. Skaret E, Berg E, Raadal M, Kvale G. Reliability and validity of the Dental Satisfaction Questionnaire in a population of 23-year-olds in Norway. Community Dent Oral Epidemiol. 2004;32(1):25-30.42. Brennan DS, Gaughwin A, Spencer AJ. Differences in dimensions of satisfaction with private and public dental care among children. International Dental Journal. 2001;51(2):77-82.43. Mascarenhas AK. Patient satisfaction with the comprehensive care model of dental care delivery. J Dent Educ. 2001;65(11):1266-71.44. Chapko MK, Bergner M, Green K, Beach B, Milgrom P, Skalabrin N. Development and validation of a measure of dental patient satisfaction. Med Care. 1985;23(1):39-49.45. Department of Health. Dental Quality and Outcomes Framework. UK,; 2016.46. National Institute for Health and care Excellence. Oral Health in Care Homes: Quality standard. England. 2017.47. National Institute for Health and Care Excellence. Oral Health Promotion in the Community: Quality Standard. 2016.48. NHS England. Dental Assurance Framework. England: NHS England; 2014.49. Dental Quality Alliance User Guide for Adult Measures Calculated using administrative Claims data. 2018. [cited 03/04/2018]. [Available from: https://www.ada.org/~/media/ADA/Science%20and%20Research/Files/DQA_2018_Adult_Measures_User_Guide.pdf?la=en]50. Dental Quality Alliance. User Guide for Pediatric Measures Calculated Using Administrative Claims Data 2018 [cited 03/04/2018]. [Available from: https://www.ada.org/~/media/ADA/Science%20and%20Research/Files/DQA_2018_Pediatric_%20Measures_User_Guide.pdf?la=en]51. Dental Quality Alliance. Electronic Pediatric Measures. 2016. [cited 03/04/2018]. [Available from: https://www.ada.org/en/science-research/dental-quality-alliance/dqa-measure-activities/electronic-pediatric-measures ] .52. The Child and Adolescent Health Measurement Inititiative. Guide to Topics & Questions Asked: National Survey of Children’s Health. 2016. [cited 03/04/2018] [Available from: http://childhealthdata.org/learn/NSCH/topics_questions/2016-nsch-guide-to-topics-and-questions.53. The Child and Adolescent Health Measurement Inititiative. Guide to Topics & Questions Asked: National Survey of Children with Special Health Care Needs, 2009/2010. 2010. [cited 05/04/2108]. [Available from: http://childhealthdata.org/learn/NS-CSHCN/topics_questions/3.3.0-2009-10-national-survey-of-cshcn--topics-questions.]54. National Network for Oral Health Access. The Dental Dashboard. 2015. [cited 03/04/2018] [Available from: http://www.nnoha.org/resources/dental-dashboard-information/.]55. Agency for Healthcare Research and Quality. Patient Experience Measures for the
18
CAHPS® Dental Plan Survey Rockville, MD. 2011. [cited 03/04/2018] [Available from: https://www.ahrq.gov/cahps/surveys-guidance/dental/about/survey-measures.html.]56. Australian Council on Healthcare Standards. Australasian Clinical Indicator Report. 18th ed. Sydney, Australia2017.57. California Department of Health Care Services. FFS Performance Measures California. 2018.[cited 03/04/2018] [Available from: http://www.dhcs.ca.gov/services/Pages/FFSPerformanceMeasures.aspx.]58. Indian Health Service. GPRA/GPRAMA resource guide. 2013. [cited 03/04/2018] [Available from: https://www.ihs.gov/california/tasks/sites/default/assets/File/GPRA/GPRAResourceGuide_v2.pdf.]59. Health Resources & Services Adminstration. HIV/AIDS Bureau Performance measures. 2017. [cited 03/04/2018] [Available from: https://hab.hrsa.gov/sites/default/files/hab/clinical-quality-management/oralhealthmeasures.pdf.]60. Anderson JR, editor HRSA’s Oral Health Quality Improvement Initiative. National Oral Health Conference; 2010; St Louis MO. Conference Presentation. [cited 03/04/2018] [Available from: http://www.nationaloralhealthconference.com/docs/presentations/2010/Jay%20Anderson%20-%20Improving%20Oral%20Healthcare%20in%20Safety%20Net%20Setti.pdf]61. National Quality Measures Clearinghouse. Availability of services: the number of dental providers who have provided any dental procedure to at least one child, per 1,000 eligible children. Rockville MD: Agency for Healthcare Research and Quality (AHRQ); 2015 [cited 03/04/2018] [Available from: https://www.qualitymeasures.ahrq.gov/summaries/summary/49899/availability-of-services-the-number-of-dental-providers-who-have-provided-any-dental-procedure-to-at-least-one-child-per-1000-eligible-children?q=dental.]62. Medicaid. 2700.4 Instructions for Completing Form CMS-416: Annual Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) Participation Report 2017 [cited 03/04/2018] [Available from: https://www.medicaid.gov/medicaid/benefits/downloads/cms-416-instructions.pdf.63. Association of State & Territorial Dental Directors. MCH Title V National Performance Measure for Oral Health Details and Recommended Actions. Reno; 2015 [cited 03/04/2018] [Available from: http://www.astdd.org/docs/mch-npm-combined-summary-and-detailed-overview-03-20-2015.pdf.]64. The Health Resources and Services Administration’s Health Disparities Collaborative. Oral Health Disparities Collaborative Implementation Manual 2012 [cited 03/04/2018] [Available from: http://www.nnoha.org/nnoha-content/uploads/2013/09/OHDC-Implementation-Manual-with-References.pdf.]65. Snyder J. Quality Measurement Models 2015 [cited 03/04/2018] [Available from: http://www.ada.org/en/science-research/dental-quality-alliance/2015-dqa-conference.66. National Care Quality Alliance. Annual Dental Visits 2017 [cited 03/04/2018] [Available from: http://www.ncqa.org/report-cards/health-plans/state-of-health-care-quality/2017-table-of-contents/dental.]67. Denplan. Denplan Excel Patient Survey 2016. [cited 03/04/2018] [Available from: https://www.hilltondentistry.co.uk/pdfs/excel-patient-survey-questionnaire.pdf.]68. Denplan. Excel Practice Assessment Survey. 2016.
19
69. Dental Quality Alliance. Procedure Manual for Performance Measures Development and Maintenance. American Dental Association; 2016.70. Parasuraman A, Zeithaml VA, Berry LL. A Conceptual-Model of Service Quality and Its Implications for Future-Research. J Marketing. 1985;49(4):41-50.71. Donabedian A. The seven pillars of quality. Arch Pathol Lab Med. 1990;114(11):1115-8.72. Maxwell RJ. Dimensions of quality revisited: from thought to action. Quality in Health Care. 1992;1(3):171-7.73. Newsome PR, Wright GH. A review of patient satisfaction: 1. Concepts of satisfaction. Br Dent J. 1999;186(4 Spec No):161-5.74. Brocklehurst P, Price J, Glenny AM, Tickle M, Birch S, Mertz E, et al. The effect of different methods of remuneration on the behaviour of primary care dentists. Cochrane Database Syst Rev. 2013(11):CD009853.
20
21
Tables
22
Measure set name and abbreviation
Key Reference Further references using measure
Description of measure contents Items (n)
Type of Measure
Internal consistency (Cronbach’s Alpha α)
Test-Retest reliability
Domain IOM Dimensions of Quality assessed
Dental Management Survey Brazil (Dimension 6) (DMS-BR)
Gonzales 2017(22)
N/A Self-assessment tool for use by Dentists and Practice managers to assess the quality of safety and organisational aspects of dental care delivery
6 Practice assessment tool
α = 0.632 Intraclass correlation Coefficients = 0.93 and 0.94
Structure Safety, Efficiency
Survey of Organisational Aspects of Dental Care (SOADC)
Goetz 2016(23) N/A Self-assessment tool of structural elements of the delivery of dental care, with focus on teamwork, leadership and the implementation of change within a practice
20 Practice assessment tool
α = 0.775 Intraclass correlation Coefficients = 0.732
Structure Safety, Patient Centeredness,
Dental patient feedback on consultation skills (DPFCS)
Cheng et al 2015(24)
Wong 2017(25) Patient satisfaction scale on the quality of information provided by the dentist to patients in consultations and the atmosphere of trust generated.
16 Patient Satisfaction Survey
α = 0.94 Intraclass correlation Coefficients = 0.89
Outcome Patient centeredness,
Tool Developed from Parasurman and Zalathml Construct of quality
Bahadori 2015(26)
N/A Patient satisfaction scale of the structures and processes of primary dental care. Focus on the settings in which dental care is delivered and communication between dentists and patients.
30 Patient Satisfaction Survey
α = 0.71-0.91 Not reported Structure, Process
Effectiveness, patient-centeredness, timeliness, efficiency
Burdens in Prosthetic Dentistry Questionnaire (BiPD-Q)
Reissman 2013(27)
Hacker 2015(28) Patient satisfaction scale of the perceived burdens of the processes of dental treatment during prosthetic dental procedures.
25 Patient Satisfaction Survey
α = 0.87 Not reported Process Patient-centeredness,
Burdens in Oral Surgery Questionnaire (BiOS-Q)
Reissman 2013(29)
N/A Patient satisfaction scale of the perceived burdens of the processes of dental treatment during oral surgical procedures.
16 Patient Satisfaction Survey
α = 0.84 Intraclass correlation Coefficients = 0.90
Process Patient centeredness
Tool after ‘Consensus workshop for selecting essential Oral health indicators in Europe’.
Kikwilu 2009(30)
N/A Patient satisfaction scale of the perceived quality of the setting of delivery of dental care and perceptions of treatment quality and communication.
11 Patient Satisfaction Survey
α = 0.849 Spearman rank correlation coefficients = 0751-0.923
Outcome Effectiveness, timeliness
Tool developed from Consumer assessment of Healthcare Providers and systems
Keller 2009(31) N/A Patient satisfaction scale of the perceived quality of information, communication and dental care received by dental plan holders.
23 Patient Satisfaction Survey
α= 0.74 Not reported Outcome Timeliness, effectiveness
Quality from the Larrsson Patient Satisfaction scale regarding 10 Patient α = 0.83 and Not reported Outcome Effectiveness,
23
Patient's Perspective Questionnaire
2005(32) the communication, information given, and environment of care deliver.
Satisfaction Survey
0.84 patient centeredness
Dental Visit Satisfaction Scale (DVSS)
Corah and O'Shea 1984(33)
Olausson 2016(34)Sun 2010(35)Hakeberg 2000(36)Stouthard 1992(37)
Patient Satisfaction scale, communication of oral health, rapport with dentist and comfort during treatment.
10 Patient Satisfaction Survey
α = 0.92 Not reported Outcome Effectiveness, patient centeredness
Dental Satisfaction Questionnaire (DSQ)
Davies and Ware 1981(38)
Lee 2015(39)Milgrom 2008(40)Skaret 2004(41)Brennan 2001(42)Mascarenhas 2001 (43)Chapko 1985 (44)
Patient Satisfaction scale, assessing ease of access, communication and thoroughness of care.
10 Patient Satisfaction Survey
α = (from Chapko) = 0.46-0.78
Not reported StructureOutcome
Timeliness, patient centeredness, effectiveness
Table 1 Contents of each measure set identified in the peer-reviewed literature Outcomes of internal validity and reliability testing of the measure sets identified in the peer-reviewed literature
24
Table 2: Qualitative description of measure sets identified from the grey literature summarising evidence of validation, and categorisation of measure sets by domains and dimensions of
quality
Measurement collection Items (n)
Description of Measure Domains Dimensions Validation
Dental Quality and Outcomes Framework (45)
15 Measures of patient satisfaction of their dental state and dental practice, clinical effectiveness and patient safety.
Process,Outcomes
SafetyEffectivenessPatient centerednessTimeliness
Not reported
NICE oral Health in care home(46) 9 Measure of the oral health needs of nursing and care home residents and the provision of care.
Structure, Process, Outcome
Patent centerednessEffectivenessEquity
Not reported
NICE Oral Health Promotion in the community (47)
15 Measures of access to health promotion resources within the community.
Structure, Process, Outcome
Patient CenterednessEffectivenessEquity
Not reported
Dental Assurance Framework Policy(48) 12 Claim data-based assessment of Provision of fluoride varnish, sealants and radiographs, rate of extractions, endodontics, patient reattendance and patient satisfaction.
Process,Outcomes
EffectivenessPatient CenterednessTimeliness
Not reported
Dental Quality Alliance Adult Measures(49) 3 Process indicators of the evaluation and ongoing care of patients with periodontitis and provision of topical fluoride in patients with elevated risk.
Process Effectiveness Face validity gained through consensus of members. Data element and convergent validity testing undertaken
Dental Quality Alliance Paediatric measures(50)
12 Measures of the utilisation of services; provision of sealants, fluoride, prevention, treatment; continuity of care; emergency department visits and follow up; cost.
Process, Outcomes
Effectiveness RAND-UCLA method used to gain consensus of face validity of measure concept. Data element collection validity assessed with Kappa statistics
Dental Quality alliance Electronic Paediatric measures (51)
2 Measures of utilisation of preventive and treatment services.
Process Effectiveness Face validity gained through consensus of members. Data element and convergent validity testing undertaken
Child and adolescent Health Measurement Initiative National Survey of Children’s Health(52)
3 US national survey. Measures of utilisation of treatment services, preventive services and presence of toothache, bleeding gums, decay and cavities.
Process, Outcomes
Effectiveness Not reported
Child and adolescent Health Measurement Initiative National Survey of Children with Special Health Care Needs, 2009/2010 (53)
2 US national survey, assessment of need and utilisation of preventive services.
Process Effectiveness Not reported
National Network for Oral Health Access Dental Dashboard(54)
15 Practice based dashboard to measure caries at recall, risk assessment; provision of sealants, topical fluoride, self-management goal setting and review, completion of treatment plans, recall rates,
Structure, Process and outcome
EffectivenessEfficiencyPatient Centeredness
Not reported
25
recommendations, and practice finances.Agency for Healthcare Research and quality Patient Experience Measures for the CAHPS® Dental Plan Survey (55)
21 National patient survey to assess patient’s assessment of care from dentist and staff, access to dental care, dental plan costs and services and patient satisfaction.
Outcomes Patient-centerednessTimeliness
Not reported
Australian Council on Healthcare Standards(56)
13 Measures of use of radiographs in new patients, retreatment rates, extraction of deciduous teeth, complications following extractions.
Process, Outcomes
Effectiveness Not reported
California department of Health Care Services(57)
15 Measures of the use of preventive services, sealants, fluoride varnish, treatment services, continuity of care.
Process Effectiveness Not reported
Indian Health Service (58) 3 Measure of receipt of topical fluoride dental sealants and access to oral health care.
Process, structure Effectiveness Not reported
HRSA HIV/AIDS bureau performance measures(59)
5 Measures of provision of oral health education, periodontal screening, treatment planning and completion and taking dental and medical history in HIV patients.
Process Effectiveness Not reported
HRSA Oral Health Quality Improvement initiative (60)
9 Treatment plan completion, use of services, provision of oral health education, Sealants, fluoride, periodontal screening.
Process Effectiveness Not reported
Q-METRIC(61) 1 Measure of availability of services. Structure Effectiveness Not reportedCMS-146 Measures(62) 7 Measures of use of dental services, preventive
services, treatment services, sealant.Process Effectiveness Not reported
MCH Title V National Performance Measure for Oral Health Summary (63)
3 Measure of percentages of: children with decay/cavities, pregnant women receiving dental care and children receiving preventive dental care.
Process Effectiveness Not reported
Oral Health Disparities Collaborative Pilot Measures (64)
17 Measures of rates of perinatal and early childhood caries, treatment plan construction and completion, paediatric dental exam and treatment plan. Fluoride varnish application an, continuity of care and fluoride assessment.
Process EffectivenessTimeliness
Not reported
Permanente Dental associates(65) 32 Measures of: Use of fluoride, sealants, clinical incidents, exam rate, continuity of care, specialist care referral, % of specialty care completed by general dentist.
Process, Outcomes
EffectivenessPatient CenterednessSafetyEfficiency
Not reported
NCQA 2017 State of Health Care Quality HEDIS measure Annual dental visits(66)
1 Measure of Medicaid members who attended for a dental visit.
Process Effectiveness Not reported
Denplan Excel Patient Survey(67) 12 Patient reported outcome measures of satisfaction with their dental health and satisfaction with their dental care provision.
Outcomes Patient Centeredness Not reported
Denplan Excel Quality programme(68) 129 122 Checklist style questions, rating of aspects of the structures and processes of dental care delivery against quality standards. 7 Percentage measures of
Structure, Process SafetyEffectivenessEfficiency
Not reported
26
process.
27
28
Table 3: Classification of all Peer Reviewed and Grey Literature measures identifed according to Dimensions and Domains
Domains
Structure Process Outcome
Safe
Dentist: Nurse Ratio Recording of medical history Patient satisfaction with cleanliness of facilities
Evidence of staff training/certification Evidence of incident reporting being carried out Patient satisfaction with quality improvement initiatives
Evidence of ensuring that suppliers/contractors are certified Number of serious incidents
Building set up to allow decontamination away from clinical areas
Evidence of Complaints handling procedures
Evidence of Data protection and handling procedures
Cleanliness of practice
Evidence of practice infection control measures
Use of single use equipment where feasible
Certification of buildings and surgery safety
Evidence of Medical emergency equipment
Effective Percentage of patients receiving oral health exam Patient rating of comfort in daily function
Percentage of patients receiving soft tissue screening Patient rating of comfort during visit
Percentage of patients receiving emergency treatment Patient rating of ease to eat
Percentage of patients receiving planned treatment Patient rating of appearance of teeth
Percentage of patients receiving preventive advice Patient rating of comprehensiveness of examination/treatment
Number of patients having radiographs taken Patient satisfaction with treatment received
Percentage of patients receiving dental follow-up after emergency department visit for dental cause Patient rating of quality of treatment
Extraction to endodontics ratio Periodontal health: Number of sites bleeding on probing
Evidence of Caries and perio risk assessment being carried out Caries: Number of decayed teeth
Percentage of patients receiving treatment for caries Caries: Prevalence of Early childhood caries
Prevention: fillings ratio Caries: New caries at recall of patient
29
Number of referrals to medical care Number of patients that are caries free
Number of referrals to secondary dental care Plaque on children’s teeth
Percentage of patients with a recording of BPE Referral to secondary care for paediatric Tooth extraction
Percentage of patients having comprehensive perio exam Emergency department visits from dental related cause
Percentage of patients with history of periodontitis undergoing course of periodontal therapy Patient reported Oral Health Related Quality of Life
Percentage of patients with fluoride needs assessment Proportion of endodontic teeth that required retreatment
Percentage of patients receiving preventive advice Proportion of sealants that require retreatment
Provision of Fissure sealants in high risk groups Number of deciduous teeth extracted
Provision of Fluoride therapy in high risk groups Complications following treatment
Extractions following endodontics
Proportion of fillings that subsequently required retreatment/endodontics/extraction
Longevity of restorations
Patient-centred Percentage of patients with named regular dentist Evidence of assessment of needs Patient satisfaction with communication – listening to patient
concerns
Access to hygienist Development of personalised treatment plans Patient satisfaction with communication – showing concern
Access to Out of hours care Setting self-management goals for patients Patient satisfaction with communication – explaining treatments
Comfort of dental practice Time spent with patients Patient satisfaction with communication – treatment/preventive advice
Aesthetics of dental practice Percentage of patients seeing same dentist/dental team at consecutive visits
Patient satisfaction with communication – giving appropriate level of information
Ease of payment Percentage of patients having treatment by their regular dentist Patient perception of dentist’s acceptance of them as a personPercentage of patients that are able to see their own dentist for emergency treatment Patient satisfaction with courtesy and respect of dental team
Extractions/endodontics completed by patient’s general dentist Patient satisfaction with time spent with dental team
Length of treatment sessions (comfort to patient) Patient satisfaction with helpfulness of staff
Patient rating of ‘atmosphere’ of dental environment
Percentage Patients that would recommend to friend
Patient rating of trust
Patient satisfaction of dental team’s ability to respond to their needs
30
Patient satisfaction with written information
Patient satisfaction with dentist
Patient rating of comfort of treatment
Patient reported pain
Patient retention – number of patients who stay with practice over time period
Timely
Percentage of group of interest who have access to dentist Timeliness of treatment plan completion Patient satisfaction with waiting times – to get standard
appointmentPercentage of group of interest that have access to oral health education Timeliness of administrative claims Patient satisfaction with waiting times – to get emergency
appointment
Patient satisfaction with waiting times – in surgery
Proportion of high risk patients who have been able to access dentist
Efficient
Generalist: specialist ratio in primary care Patient satisfaction with cost
Quality of Interpersonal relationships between members of dental team Patient rating of structure of dental appointments
Dental team satisfaction with their leadership Percentage Percentage of treatment plans completed
Dental team satisfaction with their ability to make changes Number Patients failing to attend
Responsiveness of practice and team to making changes Percentage of patients reattending within 3 months
Stress of team members within dental practice Average Cost of treatment per patient
Use of modern equipment Number of Dental encounters/hour
Longevity of restorations
Equity
Evidence of local arrangements to assess health needs Percentage of group of interest that receive preventive advice/treatment in non-dental setting Patient perception of dentist’s acceptance of them as a person
Local arrangements to identify of high needs groups Patient rating of ease of access
Local arrangements to ensure access for high needs groups
Ease of access – car parking, disabled access
31
Figure Legends
: Figure 1 PRISMA Flow diagram of literature screening and identification process
Figure
32
