CLINICAL SCENARIO

Your practice has attracted more and more patientswho are concerned about caries and the effect of oralhygiene materials being presented as preventative.While attending a dental meeting, you pass a commer-cial display advertising what appears to offer com-pelling evidence for the effectiveness of chlorhexidinetreatment as a preventative medicament. The sales rep-resentative offers you a list of studies to support theapplications for this material; however, the list is muchtoo long for you to read every article. Instead, youdecide to search the literature yourself for an overviewarticle on chlorhexidine.

A comprehensive structured review of the publishedliterature, that has an explicit and focused question,rules for inclusion of primary studies to present as evi-dence, an explanation of the strength of the evidence,and a summary of the collective findings from the pri-mary studies is called an overview. When the data fromthese primary studies allow combining in rigorous sta-tistical analyses it is called a meta-analysis.

Overviews of the scientific literature are very usefulto clinicians and researchers who are responding toquestions on the efficacy of patient therapy, or formu-lating a research project to address a basic science ortransitional research question related to dentistry.1-3

An overview that addresses a well-defined question ismore helpful to a reader than a haphazard or biasedselection of the available citations.

Overviews must be read as critically as a primaryresearch article. An overview uses specific method-ologic criteria and should be viewed “as a study initself.” It must pose a question, gather criteria-baseddata on the question from published studies of thehighest quality, analyze the data, and then draw con-clusions from the analysis. The fundamental differencebetween an overview and primary articles reportingthe results of a study is the unit of analysis, not the sci-entific principles that apply.

STRATEGIES FOR ASSESSING ANOVERVIEW

It is extremely important for the reader to follow aspecific strategy when assessing an overview article oran article that incorporates a meta-analysis in thereview. The basis elements of such a strategy shoulddetermine whether: (1) a clearly defined question wasaddressed, (2) specific search strategies were used, (3) preparation of the results was detailed, (4) theoverall results of the overview were appropriate, and(5) the results can be applied to patient care.

Unless an overview clearly states a question, thereader has no way of knowing if the overview is rele-vant to a particular treatment option the clinician maybe evaluating. The question in the overview must have3 basic parts: (1) what is being reviewed (patient histories,elements of the physical examination, diagnostic tests,treatment modalities, treatment outcomes), (2) in whom(the experimental population of interest), and (3) forwhat outcomes (consequences).

The overview should provide an explicit statement ofthe search strategies used by the author. Such strategiesshould include, but not necessarily be limited to, one ormore bibliographic databases such as PUBMED, MED-LINE, or GREATFUL-MED, and include a statementof which key words were used and in what order theywere used. The criteria used to select studies for anoverview should be as rigorous as the criteria used toinclude or exclude subjects in any clinical study. If infor-mation is lacking on how the search was conducted, thequality of the search is in question and the credibility ofthe overview is suspect. Methods that enabled theauthor to search for classical articles, such as those foundin the Index Medicus, should also be used. A system forsearching the references of relevant articles for other ref-erences that may have an impact on the experimentalquestion should be included. Having 2 or more peopleparticipate in selecting articles for inclusion in anoverview guards against bias and errors. If there is goodagreement between the 2 participants, then the readerwill have more confidence in the results of the overview.

A listing of inclusion/exclusion criteria used toselect articles must be included. Precise criteria suchas experimental or clinical outcomes, treatmentmaneuver, physical or experimental tests, histories,or prognostic factors are important. In addition, keyelements in the study design, such as the use of ran-domized clinical trials, case series, or cohort studiesare examples of “quality filters” authors should use inidentifying the inclusion or exclusion criteria.

EVIDENCE-BASED DENTISTRY SERIES

The majority of the information contained in this article has beenadapted from materials developed by the Evidence-Based Med-icine Working Group, Department of Clinical Epidemology,McMaster University, Hamilton, Ontario, Canada.

aProfessor and Chair, Department of Prosthodontics, School of Den-tistry, University of North Carolina, Chapel Hill, N.C.

bProfessor Emeritus, Department of BMS/Prosthodontics, School ofDentistry, University of Michigan, Ann Arbor, Mich.

JULY 2000 THE JOURNAL OF PROSTHETIC DENTISTRY 17

The overview: An article that interrogates the literature

David A. Felton, DDS, MS,a and Brien R. Lang, DDS, MSb

Searches involving languages other than English arecritical. Personal contact with experts in the field ofresearch addressed by the overview also should not beoverlooked. There are 2 important reasons for usingpersonal contacts. The first is to identify publishedstudies that might have been missed. The second is toidentify unpublished studies. Although the inclusionof unpublished studies is controversial, their omissioncan lead to “publication bias” because there is a high-er likelihood that only studies with “positive” resultsare published. If unpublished studies are included inan overview, they should be handled in the same wayas published studies. Full written reports should beobtained, and the validity of both published andunpublished studies should be appraised.

Accurate assessment of the quality of the methodsused in the primary studies cited is essential for cor-rectness of the conclusions drawn in the overview. Forexample, a number of case series and cohort studiessuggested that occlusal contacts were maintained andwear was not a problem after placing Class II compos-ite restorations in maxillary premolars. This served asjustification to conduct a larger randomized clinicaltrial that demonstrated wear resistance was not effec-tive in proximal/occlusal restorations. The results ofan overview that included only the case series andcohort studies lead to an incorrect conclusion thatrestoring teeth with proximal/occlusal defects usingcomposite materials was acceptable. This emphasizesthe need to use the strength of the randomized clinicaltrial research design for questions related to therapy.

The reader can only make decisions about the qual-ity of a primary study when the study methods arereported in sufficient detail. The reader must be ableto determine whether the study met minimal scientificcriteria that would allow strong inferences to be drawnfrom the results. If for example, the conclusions drawndiffer significantly from other cited studies, can thesedifferences be explained on the basis of the method-ologic quality? The truth most likely lies in the con-clusions drawn from the studies in which the method-ologic quality was not questionable and the patientpopulation/subjects and outcome measurements inthe included studies were similar. However, the readermust recognize that differences could also be due todifferences in subjects in the primary study or differ-ences in the exact outcome measurement used in thatparticular study.

Before scrutinizing how the overview author com-bined the results of the individual studies, the readermust be satisfied that it makes sense to combine thisset of studies. Are the patients, treatments, outcomes,and research methods sufficiently similar that it makesclinical and biologic sense to combine them? If not,hopefully the overview author refrained from trying tocombine the results statistically. The author must state

explicitly the basis for any conclusions reached, andprovide a unifying explanation for conflicting results.If the overview of dissimilar studies combines theirresults, the reader is cautioned about accepting theconclusion.

Some overviews are initiated to settle controversy.Therefore, it should come as no great surprise that theprimary articles included in the overview would differand have opposing results. In a number of instances,individual articles may agree on the effect of a factorbeing examined in an experimental question, but dis-agree on the magnitude of the effect. Good overviewsconfront these differences and try to explain them.Differences in study results generally arise from 5sources: (1) different sorts of patients (with differentoral environmental conditions or responsiveness totreatment), (2) different histories, ways of performingthe diagnostic tests, or performing treatments (includ-ing the extent of the therapy, lengths of the test peri-ods, combinations of treatment, and compliance), (3)different outcomes (defined and measured in differentways and with different measurement systems ordegrees of sensitivity), (4) different study methods (withdifferent rigor and power), and (5) the play of chance.

All 5 of these possible explanations should havebeen considered by the author in interpreting individ-ual study results to ensure that it makes clinical senseto combine the studies. In addition, it is possible totest the extent to which differences among results ofindividual studies are “significant” (greater than couldbe expected due to chance alone). The statistical analy-ses that are used to do this are called tests of homo-geneity. The more significant (closer to zero) the test,the less likely it is that the observed differences are dueto chance alone. When there is “statistically signifi-cant” heterogeneity (a low probability of the differ-ences being due to chance alone), both the “average”effect and the confidence interval around the averageeffect need to be interpreted cautiously.

Unfortunately, tests of homogeneity have limitedpower to detect differences. So, a nonsignificant testdoes not necessarily rule out important differences,and large differences between study results still dictatesome degree of caution in interpreting the overall find-ings. At the same time, conclusions that are drawn onthe basis of between study comparisons should beviewed skeptically. Even when there are large differ-ences between the results of different studies, a sum-mary measure from all the best available studies mayprovide a better estimate for clinical use than theresults of any 1 study. However, the strength of infer-ence associated with the overall effect is weaker thanwhen study results are consistent.

The results of individual primary studies should bereported in sufficient detail that the reader is able tocritically assess the basis for the reviewers’ conclusions.

THE JOURNAL OF PROSTHETIC DENTISTRY FELTON AND LANG

18 VOLUME 84 NUMBER 1

Tables summarizing crucial aspects of methods andresults can be helpful in quickly deciding whether con-clusions really are consistent with data. Clinical andstatistical significance should receive appropriate atten-tion in the overview.

In clinical research, investigators collect data fromindividual patients. In overviews, investigators collectdata from individual studies. These data must also besummarized, and investigators are using quantitativemethods to do so. Simply comparing the number of“positive” studies to the number of “negative” studiesis not an adequate way to summarize the results, for itis possible for a study to be counted as “positive” inone overview and “negative” in another. There is alsoa tendency to overlook small but clinically importanteffects when studies are statistically “nonsignificant”(but potentially clinically important) results are count-ed as “negative.”

Typically, meta-analyses weigh studies according tothe precision of their results, that is, the overall resultsrepresent a weighted average of the results of the indi-vidual studies. Occasionally, studies are also givenmore or less weight depending on their quality. Ifthere are large differences in the quality of the studies,poorer quality studies may be given a weight of zero(excluded) either in the primary analysis or in a “sen-sitivity analysis.” In such a sensitivity analysis, theinvestigators would leave out the poor quality studiesfrom the analysis, and find out whether that made animportant difference in the overall estimate.

Sometimes the outcome measures that are used indifferent studies are similar but not exactly the same.For example, different trials might measure wear resis-tance of several different composites. If the subjects inthe different studies and the interventions are reason-ably similar, it might still be worthwhile to estimatethe average wear resistance of the composite materialsacross studies. One way of doing this is to summarizethe results of each study as an “effect size.” The effectsize is the difference in wear between the differentcomposites and the control groups (usually amalgam)divided by the standard deviation. Hence, the resultsof each study are summarized for the number of stan-dard deviations of difference between the compositesand control groups. In such a situation, it is possible tocalculate a weighted average of effect sizes from studiesthat measured an outcome of interest (wear resistance)in different ways. However, it is difficult to interpretthe clinical importance of an effect size. Meta-analysesthat use an effect size to summarize the results shouldalso report the results in a way that clearly conveys prac-tical importance (for example, by translating the sum-mary effect size back into natural units).

As of April 2000, a total of 767 review articles indentistry have been reported, but only 64 meta-analy-ses have been published in dental journals and cited in

PUBMED. “Meta-analysis is the retrospective system-atic identification, appraisal, and comparison of allclinical studies relevant to a selected topic of interest,using statistical methods to combine and summarizeresults from the related studies.”4 It is perhaps the bestmethod for minimizing bias from related investiga-tions. Clinical trials often have small sample sizes thatpreclude the use of strong statistical analysis of theresults. Combining the results of multiple similar trialsusing meta-analysis may increase the statistical powersufficiently to allow detection of small but significanteffects of therapy that the individual trials could notdetermine.

One of the advantages of an overview is that adiverse range of patients is usually included across allstudies reviewed, and if the results are consistent acrossstudies, they apply to this wide variety of patients.However, the clinician may still be left with doubtsabout the applicability of the results. For example, thesubject in our study may present with only maxillarypremolar teeth, a slightly different situation than any ofthe subjects included in the trials on the wear of differ-ent composite materials against natural posterior teeth.One might question whether the composites have alarger effect on molar teeth than premolar teeth.

Although it is a good idea to look for focusedoverviews because they are more likely to provide validresults, this does not mean that you should ignore out-comes that are not included in an overview, whichwould be important to the subjects, such as esthetics.When making a clinical decision, the expected benefitsmust be weighed against the potential harms and costs.Although this is most obvious for deciding whether touse a therapeutic or preventive intervention, it is impor-tant to remember that providing patients with informa-tion about causes of dental disease or prognosis can alsohave both benefits and harms.

SEARCHING FOR AN “OVERVIEW”

The National Library of Medicine’s PUBMED cita-tion listing can be accessed through its Internetaddress: www.ncbi.nlm.nih.gov/PUBMED. The useof the PUBMED system requires some basic knowl-edge of the use of key words to allow access to the arti-cles of interest, along with the skills to properly limitthe scope of the search without eliminating useful ref-erences. For example, a query of the PUBMED systemfor articles related to the use of chlorhexidine as a pre-ventative treatment for caries is summarized in Table I.

The busy clinician does not have the time to review200+ citations to determine whether he/she shouldconsider the use of chlorhexidine as a treatmentmodality for caries reduction in his/her dental prac-tice. Unfortunately, the use of Reviews in the searchquery may also fail to provide the reader with a suffi-cient analysis of the articles cited in the Review manu-

FELTON AND LANG THE JOURNAL OF PROSTHETIC DENTISTRY

JULY 2000 19

script to allow careful interpretation of the data pre-sented. For example, in Table I, the 3 articles notedunder Query 35-7 did not provide the references byvan Rikjom et al8 discovered in Query 6, which was ameta-analysis on chlorhexidine therapy.

In the overview by van Rijkom et al,8 the authorsstated that they would assess the caries-inhibiting effectof chlorhexidine treatment. Specifically, the authorswould examine factors that potentially could modify theeffect of chlorhexidine in caries prevention, that is, theapplication method, application frequency, target popu-lation, fluoride regime, tooth surface involved, andcaries diagnosis. Although the authors identified “whatwas being reviewed” and for “what outcomes” they didnot specify “in whom” in their study objectives.

The authors used the MEDLINE database to begintheir search of the literature for articles on the effect ofchlorhexidine treatment in caries prevention. Thesearch was limited to articles that appeared in printbetween the years 1975 and 1994 in English, French,or German. Twenty-four articles were found in thesearch. Two examiners using comparability for dentaland methodologic reasons independently reviewed thearticles. The 2 examiners subjected the 24 articles tothe criteria of: (1) chlorhexidine applied to permanentteeth from 11- to 15-year-old children; (2) studiesperformed in clinical trials with randomly assignedtreatment groups, including generally treated experi-mental groups as well as treatment focused on subjectsin the experimental group with Streptococcus mutans> 25 × 105/mL, saliva; (3) the availability of cariesincidence data; (4) a treatment duration of at least 1year; and (5) evaluation at the end of the treatmentperiod. These “quality filters” were used to identifythe “in whom” part of the experiment question, andto the narrow overview, and identification of potentialfactors that could modify the effect of chlorhexidine.Using the criteria, a total of 10 articles were eliminat-ed from further review.

Further, the use of 2 investigators to independent-ly review the studies identified in the MEDLINEsearch assisted in the elimination of any bias in theselection of articles for inclusion. Each investigator

determined whether an article met both dental andmethodologic criteria. Studies that were out of the agerange, were not clinical trials, contained no caries inci-dence data on surfaces involved, or used treatmentthat lasted less than 1 year were eliminated from con-sideration. Studies that specified application methods,application frequencies, caries risk, fluoride regime,caries diagnostic levels, and included tooth surface wereconsidered in the overview. Studies that contained bothexperimental and control populations needed to be evi-dent.

In the van Rijkom overview, investigators describedthe caries-inhibiting effect of chlorhexidine by a pre-vented fraction calculated for each reported study. Theprevented fraction was the difference in number ofnew decayed and filled surfaces between the controlgroup and the chlorhexidine group, divided by thenumber of new decayed and filled surfaces in the con-trol group. It indicated the percentage reduction ofcaries incidence in the chlorhexidine group and wasconsidered less sensitive to experimental circumstancesthan the absolute reduction. Studies that included fol-low-up years indicating the total length of the study aswell as the treatment duration were excluded becausethe prevented fraction was assumed to be independentof the duration of the study.

To compare data across studies by meta-analysis,the investigators calculated the 95% confidence inter-vals for the prevented fraction from the data report-ed for each study. This was possible given the report-ed mean data and the standard deviations in eachstudy. The authors were able to pool data at the 95%confidence interval from the selected studies todemonstrate the homogeneity of the publishedresults. Although only 8 articles were included in theanalysis, the number of subjects in the individualreports varied from 9 to 72. Combining the articlesinto a meta-analysis allowed the assessment of 306total subjects, giving strength to the statistical analy-sis of the results. The total sample size, appropriateinclusion/exclusion criteria for the articles reviewed,95% confidence interval selected, and homogeneityof the individual study results allow the reader toclearly assess that the use of chlorhexidine in this par-ticular patient population will have a dramatic effecton the caries incidence, and thus should be applica-ble to the reader’s practice.

An important finding in the meta-analysis thatshould be of invaluable use to the reader related to the“application method” for use of chlorhexidine in thispatient population. In the 8 articles included in theanalysis, application of topical gels (professionallyapplied), rinsing with 0.5% chlorhexidine rinse, use ofchlorhexidine containing tooth pastes, or combinationsof toothpaste and rinsing, were studied. The meta-analysis demonstrated no differences in the application

THE JOURNAL OF PROSTHETIC DENTISTRY FELTON AND LANG

20 VOLUME 84 NUMBER 1

Table I. Query results in PUBMED system

Number ofQuery citations inno. Key words PUBMED

1 Chlorhexidine and caries 2362 Chlorhexidine and dental caries 2083 Chlorhexidine and caries and reviews 34 Chlorhexidine and dental caries and reviews 25 Chlorhexidine and caries and meta-analyses 26 Chlorhexidine and dental caries and 1

meta-analyses

method. This enables the clinician to tailor treatmentbased on patient compliance rather than a single pre-scribed therapy. Thus, this meta-analysis provides use-ful information for the clinician in the treatment of hisjuvenile patients.

SUMMARY

Although the question on the efficacy of chlorhexi-dine therapy for the clinician’s adult patient may nothave been totally answered, a greater perspective onthis treatment modality was gained by the readerbecause of the quality of the overview cited, and theevidence-based approach used by the authors in per-forming the overview.

REFERENCES

1. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. How to read reviews andeconomic analyses. In: Clinical epidemiology—a basic science for clin-ical medicine. Boston: Little, Brown and Company; 1991. p. 379-94.

2. Oxman AD, Guyatt GH, the Evidenced-Based Medicine Working Group.How to use an overview. In: Evidence-based medicine: users’ guides tothe medical literature. Chapter VI. Hamilton, Canada: McMaster Univer-sity Health Sciences Centre, 1993.

3. Oxman AD, Cook DJ, Guyatt GH, for the Evidenced-Based MedicineWorking Group. Users’ guide to the medical literature. VI: How to usean overview. JAMA 1994;272:1367-71.

4. Palmer AJ, Sendi PP. Meta-analysis in oral health care. Oral Surg OralMed Oral Pathol Oral Radiol Endod 1999;87:135-41.

5. Ismail AI. Prevention of early childhood caries. Community Dent OralEpidemiol 1998;26(Suppl 1):49-61.

6. Kidd EAM. Role of chlorhexidine in the management of dental caries. IntDent J 1991;41:279-86.

7. Hogg SD. Chemical control of plaque. Dent Update 1990;17:332-4.8. Rijkon HM, Truin GJ, van’t Hof MA. A meta-analysis of clinical studies

on the caries-inhibiting effect of chlorhexidine treatment. J Dent Res1996;75:790-5.

Copyright © 2000 by The Editorial Council of The Journal of ProstheticDentistry.

0022-3913/2000/$12.00 + 0. 10/8/108651

doi:10.1067/mpr.2000.108651

FELTON AND LANG THE JOURNAL OF PROSTHETIC DENTISTRY

JULY 2000 21

Bound volumes available to subscribers

Bound volumes of The Journal of Prosthetic Dentistry are available to subscribers (only) forthe 2000 issues from the publisher at a cost of $92.00 ($106.00 international) for Vol. 83(January-June) and Vol. 84 (July-December). Shipping charges are included. Each bound volumecontains a subject and author index, and all advertising is removed. Copies are shipped within30 days after publication of the last issue in the volume. The binding is durable buckram with thejournal name, volume number, and year stamped in gold on the spine. Volumes 81 and 82 arealso available. Payment must accompany all orders. Contact Mosby, Subscription Customer Ser-vice, 6277 Sea Harbor Dr, Orlando, FL 32887, or call 800-654-2452 or 407-345-4000.

Subscriptions must be in force to qualify. Bound volumes are not available in place ofa regular Journal subscription.