Chapt Research Design

8/14/2019 Chapt Research Design

1/66

Copyright 1999 by W.B.Saunders Company. All rights reserved.

Chapter 7

ClarifyingResearch

Designs


2/66


Quantitative Research

DescriptiveDescriptive

CorrelationalCorrelational

Quasi-experimentalQuasi-experimental

ExperimentalExperimental

IncreasedIncreased

ControlControl

withwithDesignDesign


3/66


Research Design

Definition: Blueprint or detailedDefinition: Blueprint or detailed

plan for conducting a studyplan for conducting a study

Purpose, Review of Literature, &Purpose, Review of Literature, &

FrameworkFramework

Study DesignStudy Design


4/66


5/66


Concepts Relevant to Design

Causality:Causality: AA BB

PressurePressure UlcerUlcer Multicausality:Multicausality:

Years SmokingYears Smoking

High-Fat DietHigh-Fat Diet

Limited ExerciseLimited Exercise

HeartDisease


6/66



ProbabilityProbability:: likelihood oflikelihood of

accurately predicting an eventaccurately predicting an event BiasBias:: slanting findings awayslanting findings away

from the truthfrom the truth

ManipulationManipulation:: implementationimplementationof a treatmentof a treatment


7/66Copyright 1999 by W.B.Saunders Company. All rights reserved.


ControlControl

Implemented throughout the designImplemented throughout the design Improve accuracy of findingsImprove accuracy of findings

Increased control in quasi-Increased control in quasi-experimental researchexperimental research

Greatest in experimental researchGreatest in experimental research



Varying Control in Studies

Quasi-experimentalQuasi-experimental ExperimentalExperimental

Lower ControlLower Control High ControlHigh Control

Type of Sample Selected:Type of Sample Selected:

Convenience ----------|--------> RandomConvenience ----------|--------> Random






Type of Sample Selected:Type of Sample Selected:

Heterogeneous -----|----> HomogeneousHeterogeneous -----|----> Homogeneous




MeasurementMeasurement

of Dependent Variableof Dependent Variable

CrudeCrude PrecisePrecise

Measurement------|----> MeasurementMeasurement------|----> Measurement




ControlControl

of Independent Variableof Independent Variable

Limited orLimited or HighlyHighly

no Control ----------|--------> Controlledno Control ----------|--------> Controlled






GroupGroup

no comparison group ----> comparisonno comparison group ----> comparison

group ----> alternative treatmentgroup ----> alternative treatmentcontrol group ----> no treatment controlcontrol group ----> no treatment control

groupgroup




Selection of GroupsSelection of Groups

No randomization ---------> RandomizationNo randomization ---------> Randomization






Setting SelectedSetting Selected

HighlyHighly

Natural -----------|----------> ControlledNatural -----------|----------> Controlled



Design Validity

Measure of truth or accuracy of aMeasure of truth or accuracy of a

study examined with critique &study examined with critique &

controlled in conducting a studycontrolled in conducting a study

Statistical conclusion validityStatistical conclusion validity

Internal validityInternal validity

Construct validityConstruct validity

External validityExternal validity



Types of Design Validity

Statistical Conclusion ValidityStatistical Conclusion Validity ----

is concerned with whether theis concerned with whether theconclusions about relationships orconclusions about relationships or

differences drawn from statisticaldifferences drawn from statistical

analyses are an accurate reflectionanalyses are an accurate reflectionof the real world.of the real world.



Statistical Conclusion Validity

Low statistical power: Type II error,Low statistical power: Type II error,

no significant difference whenno significant difference whenthere is.there is.

Violated assumptionsViolated assumptions

At least interval dataAt least interval data Random sampleRandom sample

Normal distributionNormal distribution



Statistical Conclusion Validity

Reliability of Instruments:Reliability of Instruments:

consistency in measurementconsistency in measurement Physiological measures: Precision,Physiological measures: Precision,

such as with temp., BPsuch as with temp., BP

Likert scale: Homogeneity--ReliabilityLikert scale: Homogeneity--Reliability

with Cronbachs alpha (.7, .8, .9)with Cronbachs alpha (.7, .8, .9) Observations: Stability--InterraterObservations: Stability--Interrater

reliabilityreliability


19/66



Internal Validity --Internal Validity --

is the extent to which the effectsis the extent to which the effectsdetected in the study are adetected in the study are a

true reflection of reality,true reflection of reality,

rather than the result ofrather than the result of

extraneous variablesextraneous variables..




Internal Validity

History: event impacting subjectsHistory: event impacting subjects

& data collected& data collected

Maturation of subjects: wiser, tired,Maturation of subjects: wiser, tired,

older influencing responsesolder influencing responses

Testing: learn from being in studyTesting: learn from being in study

Instruments accuracyInstruments accuracy



Internal Validity

Subject selection nonrandomSubject selection nonrandom

& potentially biased& potentially biased

Mortality: those dropping outMortality: those dropping out

of studyof study

Diffusion or imitation of treatment:Diffusion or imitation of treatment:control or comparison groupcontrol or comparison group

has access to treatmenthas access to treatment


23/66


24/66


Construct Validity

Validity of instrumentsValidity of instruments

Physiologic measure: AccuracyPhysiologic measure: Accuracy

Scales: construct validityScales: construct validity

Mono-operation bias: 1 instrumentMono-operation bias: 1 instrument

to measure a variableto measure a variable

Mono-method bias: 1 type ofMono-method bias: 1 type of

instrument, such as attitude scalesinstrument, such as attitude scales


25/66


Construct Validity

Subjects guessing outcome of studySubjects guessing outcome of study

& acting in a different way& acting in a different way

Subjects performing for experimentSubjects performing for experiment

to be seen in a good light or trying toto be seen in a good light or trying to

get an effective treatmentget an effective treatment

Treatment interactions with moreTreatment interactions with more

than one treatmentthan one treatment


26/66


External Validity---External Validity---

is concerned with the extent tois concerned with the extent towhich study findings can bewhich study findings can be

generalized beyond the samplegeneralized beyond the sample

used in the study.used in the study.



27/66


External Validity

Interactions of:Interactions of:

Selection of subjects & treatmentSelection of subjects & treatment Setting & treatment: closing clinic,Setting & treatment: closing clinic,

nonsupportive staffnonsupportive staff

History & treatment: eventsHistory & treatment: eventsinfluence treatmentinfluence treatment


28/66


Elements of Strong Design

Controlling the Environment:Controlling the Environment:

selection of the study settingselection of the study setting Natural settingNatural setting

Partially controlled setting: clinicPartially controlled setting: clinic

Highly controlled setting: labHighly controlled setting: lab


29/66



Controlling the Equivalence ofControlling the Equivalence of

Subjects & GroupsSubjects & Groups Random subjectRandom subject

selectionselection

Random assignmentRandom assignment

to groupsto groups


30/66



Controlling the Treatment (Tx)Controlling the Treatment (Tx)

Tx based on research & practiceTx based on research & practice Protocol developed for implementationProtocol developed for implementation

Document Tx was implementedDocument Tx was implemented

Checklist to determine extent TxChecklist to determine extent Txwas implemented if incompletewas implemented if incomplete

Re-evaluate Tx during studyRe-evaluate Tx during study


31/66



Controlling MeasurementControlling Measurement

ReliabilityReliability ValidityValidity

# Measurement methods# Measurement methods

Types of instrumentsTypes of instruments


32/66



Controlling Extraneous VariablesControlling Extraneous Variables

Identify & eliminate by sampleIdentify & eliminate by sample

criteria, setting, designcriteria, setting, design

Random samplingRandom sampling

Sample: heterogeneous, homogeneous,Sample: heterogeneous, homogeneous,matchingmatching

Statistical controlStatistical control


33/66


Problems with Study Designs

Inappropriate for purposeInappropriate for purpose

& framework& framework

Poorly developedPoorly developed

Poorly implementedPoorly implemented

Inadequate treatment, sample,Inadequate treatment, sample,

measurement methodsmeasurement methods


34/66


Critiquing a Study Design

1.1. Was the type of design identified?Was the type of design identified?

2.2. Was the study design linked to theWas the study design linked to thepurpose &/or objectives, questions,purpose &/or objectives, questions,

or hypotheses?or hypotheses?

3.3. Were all variables manipulatedWere all variables manipulated

or measured?or measured?


35/66



4. If the study included a treatment, was it4. If the study included a treatment, was it

clearly described & consistentlyclearly described & consistently

implemented?implemented?5.5. Were extraneous variables identified &Were extraneous variables identified &

controlled?controlled?

6.6. What were the threats to design validity inWhat were the threats to design validity inthis study?this study?


36/66



7.7. Was a pilot study performed priorWas a pilot study performed prior

to this study? What was the reasonto this study? What was the reason

for the pilot & the outcome?for the pilot & the outcome?

* Study feasibility* Study feasibility

* Refine design or treatment* Refine design or treatment

* Examine validity & reliability* Examine validity & reliability

of measurement methodsof measurement methods


37/66


Identifying a Design

in a Quantitative

Research Report


38/66


39/66


Types of DesignsTypes of Designs

Descriptive Study DesignsDescriptive Study Designs

Correlational Study DesignsCorrelational Study Designs Quasi-ExperimentalQuasi-Experimental

Study DesignsStudy Designs Experimental Study DesignsExperimental Study Designs


40/66


Identifying a DesignIdentifying a DesignIs there a treatment?Is there a treatment?

NoNo YesYes

Is the treatment tightly controlledIs the treatment tightly controlled

by the researcher?by the researcher?Is the primary purposeIs the primary purpose

examination of relationships?examination of relationships?

NoNo YesYes

CorrelationalCorrelational

DesignDesign

NoNo YesYes

ExperimentalExperimental

StudyStudy

NoNo YesYes

Quasi-ExperimentalQuasi-Experimental

StudyStudyWill a randomly assignedWill a randomly assigned

control group be usedcontrol group be used

NoNo YesYes

Is the original sampleIs the original sample

randomly selected?randomly selected?

Will the sample beWill the sample be

studied as a singlestudied as a single

group?group?

NoNo YesYes


DesignDesign


41/66


Identifying a Descriptive DesignIdentifying a Descriptive Design

Examining sequences across time?Examining sequences across time?

NoNo

Cross-sectional designCross-sectional design

NoNo YesYes

ComparativeComparative


DesignDesign

One Group?One Group?


DesignDesign

YesYes

Following same subjects across time?Following same subjects across time?

Studying events partitionedStudying events partitioned

across time?across time?CaseCase

StudyStudy

NoNo YesYes

NoNo

Data collectedData collected

across timeacross time

YesYes

Single unitSingle unit

of studyof study

NoNo YesYes

LongitudinalLongitudinal

StudyStudy

Trend AnalysisTrend Analysis Repeated measures of each subjectRepeated measures of each subject

NoNo YesYes

YesYes NoNo

Cross-sectional designCross-sectional designwith treatment partitioningwith treatment partitioning

Longitudinal design withLongitudinal design withtreatment partitioningtreatment partitioning


42/66


Descriptive Study Designs

Typical Descriptive DesignTypical Descriptive Design

Comparative Descriptive DesignComparative Descriptive Design

Time Dimensional DesignTime Dimensional Design

Case Study DesignCase Study Design


43/66


Diagramming the Design

Clarifies the variables to beClarifies the variables to be

measured or manipulatedmeasured or manipulated

Indicates focus of the study:Indicates focus of the study:

description, relationships,description, relationships,

differences, &/or testing adifferences, &/or testing a

treatmenttreatment


44/66


Diagramming the Design

Identifies data collection process:Identifies data collection process:

time for study, treatmenttime for study, treatment

implementation, measurementimplementation, measurement

of variablesof variables

Provides direction toProvides direction to

data analysisdata analysis


45/66


Typical Descriptive Design

ClarificationClarification MeasurementMeasurement DescriptionDescription InterpretationInterpretation

PhenomenonPhenomenon

of Interestof Interest

DescriptionDescription

ofof

Variable 1Variable 1


ofof



ofofVariable 3Variable 3


ofof






InterpretationInterpretation

of Meaningof Meaning

Development ofDevelopment of

HypothesesHypotheses


46/66


Comparative Descriptive Design

Group IGroup I

{variable(s){variable(s)

measured}measured}

Group IIGroup II{variable(s){variable(s)

measured}measured}

DescribeDescribe

InterpretationInterpretation


Comparison ofComparison of

Groups onGroups on

Selected VariablesSelected Variables

DescribeDescribe DevelopmentDevelopment

of Hypothesesof Hypotheses


47/66


Correlational Study Designs


Correlational DesignCorrelational Design Predictive DesignPredictive Design

Model-Testing DesignModel-Testing Design

Identif ing T pe of


48/66


Identifying Type of

Correlational Design

DescribeDescribe

relationshipsrelationships

between/amongbetween/among

variables?variables?

PredictPredict

relationshipsrelationships

between/amongbetween/among

variables?variables?

TestTest

theoreticallytheoretically

proposedproposed

relationships?relationships?


correlationalcorrelational

designdesign

PredictivePredictive

correlationalcorrelational

designdesign

ModelModel

testingtesting

designdesign


49/66


Descriptive Correlational Design

ResearchResearch

VariableVariable

11

ResearchResearch

VariableVariable

22


of variableof variable InterpretationInterpretation


Examination ofExamination of

RelationshipRelationship


of variableof variableDevelopmentDevelopment

of Hypothesesof Hypotheses



50/66


Predictive Design

ValueValue

ofof

InterceptIntercept

Value ofValue of

IndependentIndependent

VariableVariable

11

Value ofValue of

IndependentIndependent

VariableVariable

22

PredictedPredicted

Value ofValue of

DependentDependent

VariableVariable

++ ++ ==


51/66


Study Groups

Groups in comparativeGroups in comparative

descriptive studiesdescriptive studies

Control groupControl group

Comparison groupComparison group

Equivalent versusEquivalent versus

nonequivalent groupsnonequivalent groups


52/66


Pre-Experimental Designs

One-group post-testonly designOne-group post-testonly design

Post-testonly design withPost-testonly design withnonequivalent groupsnonequivalent groups

One-group pretestpost-testOne-group pretestpost-test

designdesign


53/66


Quasi-Experimental Designs

Untreated control group designUntreated control group design

with pretest & post-testwith pretest & post-test

Nonequivalent dependentNonequivalent dependent

variables designvariables design

Removed-treatment designRemoved-treatment design

with pretest & post-testwith pretest & post-test

P t t P t T t C t l


54/66


PretestPost-Test ControlGroup Design


of dependentof dependent

variablesvariables

Manipulation ofManipulation of

independentindependent

variablevariable



variable(s)variable(s)

Randomly selectedRandomly selected

experimentalexperimentalgroupgroup

PRETESTPRETEST TREATMENTTREATMENT POST-TESTPOST-TEST


control groupcontrol group

Treatment:Treatment: Under control of researcherUnder control of researcher

Findings:Findings: comparison of pretest and post-test scorescomparison of pretest and post-test scores

comparison of experimental and control groupscomparison of experimental and control groups

comparison of pretest-post-test differences between samplescomparison of pretest-post-test differences between samples

Example:Example: Your self (1990). The impact of group reminiscence counseling on a depressed elderlyYour self (1990). The impact of group reminiscence counseling on a depressed elderly

populationpopulation

UncontrolledUncontrolled testingtesting mortalitymortality

threats to validity:threats to validity: instrumentationinstrumentation restricted generalizability as control increasesrestricted generalizability as control increases

PRETESTPRETEST POST-TESTPOST-TEST

Post Test Only


55/66


Post-TestOnlyControl Group Design


of independentof independent

variablesvariables



variable(s)variable(s)


experimentalexperimentalgroupgroup

TREATMENTTREATMENT POST-TESTPOST-TEST


control groupcontrol group

Treatment:Treatment: Under control of researcherUnder control of researcher

Findings:Findings: comparison of experimental and control groupscomparison of experimental and control groups

Example:Example: Clochesy, Difani & Howe (1991). Electrode site preparation techniques: aClochesy, Difani & Howe (1991). Electrode site preparation techniques: a

follow-up studyfollow-up study

UncontrolledUncontrolled instrumentationinstrumentation

threats tothreats to mortalitymortality

validity:validity: limited generalizability as control increaseslimited generalizability as control increases

POST-TESTPOST-TEST

Ad t f Q i


56/66


Advantages of Quasi-Experimental Designs

More Practical: EaseMore Practical: Ease

of implementationof implementation

More Feasible: resources,More Feasible: resources,

subjects, time, settingsubjects, time, setting

More Generalizable:More Generalizable:

Comparable to practiceComparable to practice

Advantages of


57/66


Advantages ofExperimental Designs

More Controls: designMore Controls: design

& conduct of study& conduct of study

Increased internally valid:Increased internally valid:

decreased threats todecreased threats to

design validitydesign validity Fewer Rival HypothesesFewer Rival Hypotheses


58/66


Outcomes Research

Agencies Supporting OutcomesAgencies Supporting Outcomes

ResearchResearch

Agency for Health ServicesAgency for Health Services

Research (AHSR)Research (AHSR)

Agency for Health Care PolicyAgency for Health Care Policy& Research (AHCPR)& Research (AHCPR)


59/66


Outcomes Research

Structure of CareStructure of Care

System CharacteristicsSystem Characteristics

Provider CharacteristicsProvider Characteristics

Patient CharacteristicsPatient Characteristics


60/66


Outcomes Research

Process of CareProcess of Care

Ordering testsOrdering tests

Treatments: pharmacologicalTreatments: pharmacological

& nonpharmacological& nonpharmacological

ReferralReferral

HospitalizationHospitalization

Counseling & educationCounseling & education


61/66


Outcomes Research

OutcomesOutcomes

Clinical end pointsClinical end points Functional statusFunctional status

General well-beingGeneral well-being Satisfaction with careSatisfaction with care


62/66


Critiquing A Design

Identify elements that wereIdentify elements that were

controlled in the studycontrolled in the study

Identify possible sources of biasIdentify possible sources of bias

Are there elements that couldAre there elements that could

have been controlled to improvehave been controlled to improve

the study design?the study design?


63/66


Critiquing A Design

What elements of the design wereWhat elements of the design were

manipulated and how were theymanipulated and how were they

manipulated?manipulated? How adequate was the manipulation?How adequate was the manipulation?

What elements should have beenWhat elements should have been

manipulated to improve the validity ofmanipulated to improve the validity of

the findings?the findings?


64/66


Critiquing A Design

Based on your assessment ofBased on your assessment of

the adequacy of the design,the adequacy of the design,how valid are the findings?how valid are the findings?


65/66


Critiquing A Design

Is there another reasonableIs there another reasonable

(valid) explanation (rival(valid) explanation (rivalhypothesis) for the studyhypothesis) for the study

findings other than thatfindings other than that

proposed by the researcher?proposed by the researcher?


66/66

Questions

Documents

Chapt Research Design