Research Design and Sources of Evidence

Copyright © 2009 Wolters Kluwer Health | Lippincott Williams & Wilkins | Forrest, Miller, Overman, Newman

Evidence-Based Decision Making: A Translational Guide for Dental Professionals

Research Design and Sources of Evidence Research Design and Sources of Evidence

Chapter 3

Jane L. Forrest, EdD, RDHSyrene A. Miller, BA

Pam R. Overman, BSDH, EdD Michael G. Newman, DDS



PurposePurpose

• The purpose of this chapter is to discuss sources of scientific evidence and characteristics of research designs that constitute the evidence.

• Although EBDM emphasizes using randomized clinical trials and other quantifiable methods, this focus has evolved to include qualitative research and acknowledging that different research designs contribute to a continuum of knowledge.



ObjectivesObjectives• Identify what constitutes evidence.

• Explain the difference between research and evidence.

• Identify sources of primary and secondary evidence.

• Discuss the difference between experimental and nonexperimental research.

• Identify distinguishing characteristics of different research methods: Randomized Control, Cohort, Case Control, Case Series, and Case Report Studies.

• Discuss the difference between quantitative and qualitative research and the role of qualitative research in EBDM.

• Identify scientific sources of evidence to use in clinical decision making.



Sources of EvidenceSources of Evidence

• Scientific evidence is the product of well-designed and well-controlled research investigations that minimize sources of bias.

• Evidence is the synthesis of all valid research studies that answer a specific question.

• Evolves over time as more research is conducted, underscoring the importance of keeping current with the scientific literature



Research and the Strength of EvidenceResearch and the Strength of Evidence

• Evidence and Research often are used interchangeably.

• Evidence is distinguished from research in that a single research study does not constitute "the evidence," but rather contributes to an overall body of knowledge that has been derived from multiple studies investigating the same area.

• Evidence is considered the synthesis of all valid research studies that answer a specific question; however, in some cases, there may be only one study.



Building a Body of Scientific Evidence

Study 1

Synthesized Results

Statistical Analysis of

Synthesized Results

Reviews of Already Conducted Research

Systematic Review & Meta-Analysis

Study 2

Study 3

Study 4

Primary Primary ResearchResearch

Individual Individual Research StudiesResearch Studies

Secondary Secondary ResearchResearch

Figure 3-3. Primary vs. Secondary Research



Evidence-Based Sources: Primary & Secondary Evidence-Based Sources: Primary & Secondary

• Primary: original research publications

– Quantitative: hypothesis testing; establishing cause and effect; data reported mathematically

– Qualitative research: exploratory; why and how of decision making; data reported in narrative terms

• Secondary: filtered or synthesized publication of the primary/original research

– Systematic reviews

– Meta-analyses

– Evidence-based article reviews

– Evidence-based clinical practice guidelines



Quantitative Primary Research: Experimental StudiesQuantitative Primary Research: Experimental Studies• Researcher controls or manipulates the variables

under investigation, such as in testing the effectiveness of a treatment.

• These studies are the most complex and include randomized controlled trials and controlled clinical trials.

• The Randomized Controlled Trial (RCT) provides the strongest evidence for demonstrating cause and effect, i.e., the treatment has caused the effect, rather than it happening by chance.



Variables in Experimental Studies: RCTVariables in Experimental Studies: RCT• All experiments have at least 1 independent and 1

dependent variable.

• Concurrent enrollment of subjects and follow-up of subjects in experimental and control groups

• Assignment of subjects through random process

• Purpose of experiments is to estimate the extent to which the independent variable causes changes in the dependent variables.

• Physically manipulate independent variable– Giving experimental group new drug and control group a placebo

– Teaching 1 class with computers and 1 without



Key Characteristics of RCTsKey Characteristics of RCTs

• Ability to randomly assign subjects to either the experimental or control group and to randomly allocate treatments

• Prospective in nature and can include blind or double-blind strategies

– A double-blind RCT is one in which neither the patient nor the investigator knows whether the patient is receiving the experimental treatment or the control treatment, e.g., studies involving therapies (pills/liquids/pastes).

– More difficult to double blind studies when testing a new treatment, technique, or procedure where the investigator or patient can distinguish a difference.



Experimental ResearchExperimental Research

• Random assignment – equal chance of being in the experimental group or control group

• Double-blind – investigators or examiners are not aware of which group the subjects are assigned

• Inter-rater reliability – degree to which examiners are calibrated; achieve same results (e.g., 8 of 10 times)

• Intra-rater reliability – degree to which an examiner achieves the same results with himself



Quantitative Primary Research: Nonexperimental ResearchQuantitative Primary Research: Nonexperimental Research

• Studies in which the researcher does not give a treatment, give an intervention, or provide an exposure, i.e., data are gathered without intervening to control variables

– Includes cohort studies, case control studies, case series, and case reports

• Descriptive: used to systematically describe and interpret conditions or relationships that already exist



What Is a Cohort Study?What Is a Cohort Study?• Makes observations about the association

between a particular exposure or risk factor (tobacco) and the development of a disease (e.g., lung cancer)

– Patients/subjects have been exposed to tobacco but do not have lung cancer

• Followed over time and compared with a similar group that has not been exposed to the risk factor

• Again, a treatment/intervention is not given by the investigator.



Prospective Cohort Study DesignProspective Cohort Study Design

Exposed

UnexposedNo DiseaseDisease Disease No Disease

Persons with and without the exposureexposure of interest (e.g., tobacco) are identified at the initiation of the study. Information is then collected looking forward in time to identify disease outcomes. At the start of the study, neither group has the disease.



Guide to Research Methods http://library.downstate.edu/ebm/2100.htm

What Is a Case Control Study? What Is a Case Control Study? • Patients who already have a certain condition are compared with

people who do not

– For example, lung cancer patients are asked how much they smoked in the past

– Answers are compared with those from a matching sample without the condition

• Less reliable than either RCTs or Cohort Studies. Statistical relationship between two conditions does not mean that one condition actually caused the other.

– E.g., lung cancer rates are higher for people who earn less than $50,000 (who tend to smoke more), but this does not mean that they can reduce their cancer risk by getting a salary increase to over $50,000.



Case Control Study DesignCase Control Study Design

Disease

No Disease

UnexposedExposed Exposed Unexposed

Persons with and without the diseasedisease of interest (caries, cancer) are identified at the initiation of the study. Information is then collected looking backward in time to identify potential exposures (sugar, tobacco) that could have contributed to getting the disease.



Qualitative Primary ResearchQualitative Primary Research

• Nonexperimental in that it conducts studies in natural settings in an attempt to understand an event from the point of view of the participants

– It seeks to provide depth of understanding.

– It answers questions such as what, how, and why.

– It explores issues in more depth with those experiencing the issue, rather than testing a hypothesis to answer questions such as how many or what proportion.

• Can generate new theory



Qualitative Primary Research (cont.)Qualitative Primary Research (cont.)

• Compliments quantitative research by attempting to clarify the meaning of how many or by providing a greater understanding of why an intervention works

– Quantitative research may ask, "How many smokers have tried to quit?" whereas qualitative research explores "What stops smokers from quitting?"

– The most important consideration in designing a study is to use the right methodology to answer the question.



Table 3-1: Characteristics of Quantitative Research

Approaches6,7



Secondary Research: Systematic Reviews and Meta-AnalysisSecondary Research: Systematic Reviews and Meta-Analysis• Secondary research is filtered or synthesized publications

of the primary research literature.– Systematic reviews (SRs) and meta-analyses,

evidence-based article reviews of already conducted research, and evidence-based clinical practice guidelines

• Provides a way of managing large quantities of information by providing a summary of primary research studies that have investigated the same question

– Uses explicit predefined criteria for retrieval of studies, assessment, and synthesis of evidence from individual RCTs and other well-controlled methods



Systematic vs. Literature Reviews Systematic vs. Literature Reviews



Secondary Research: Evidence-Based Journals and Article ReviewsSecondary Research: Evidence-Based Journals and Article Reviews

• New evidence-based resources are being developed for practitioners to facilitate integrating evidence into their clinical decision making, e.g., Journal of Evidence-Based Dental Practice (JEBDP), Evidence-Based Dentistry (EBD).

• Provide concise and easy-to-read summaries of original and review articles selected from the biomedical literature based on specific inclusion criteria

• Article reviews of already conducted research often consist of a 1- to 2-page structured abstract along with an expert commentary highlighting the most relevant and practical information of the study being reviewed.



Secondary Research: Practice GuidelinesSecondary Research: Practice Guidelines• Growing source of synthesized information on a

specific topic• Guidelines are “systematically developed

statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”

• The inclusion of scientific evidence within clinical practice guidelines has now become the standard, in that guidelines should incorporate the best available scientific evidence. SRs support this process by putting together all that is known about a topic in an objective manner.



Secondary Research: Practice Guidelines (cont.)Secondary Research: Practice Guidelines (cont.)

Examples of clinical practice guidelines• ADA’s clinical recommendations on

professionally applied topical fluoride• AHA’s guidelines on antibiotic premedication• ADHA guidelines on polishing procedures



ActivitiesActivities

• Quiz

• Exercise 3-1: Type of Study Exercises



Critical Thinking QuestionsCritical Thinking Questions• Discuss how quantitative and qualitative research

are complimentary and provide an example of a study related to patient problems that would include both types of studies. [Example: how often patients floss (quantitative study) and what barriers do they encounter that prevent them from flossing every day (qualitative study)].

• Explain why a RCT is not always the appropriate research design to use.

• Provide an example of when you would first conduct a traditional literature search prior to looking for a systematic review or meta-analysis.

Documents

Research Design and Sources of Evidence