Understanding Quantitative Research + Sampling

7/29/2019 Understanding Quantitative Research + Sampling

1/72


2/72

Objec&ves UnderstandtheconceptofResearchDesign Identifythecharacteristicsofsomecommon

quantitativeresearchdesigns Defineanddiscussresearchcontrol,and

internalandexternalvalidityasthesetermsrelatetoresearchdesign

Critiqueawrittenresearchreportintermsofdesignappropriatenessandaccuracyofdesignimplementation


3/72

WhatisResearchDesign? theoverallapproachbestsuitedtoanswerspecific

questionsandachievethestatedpurposeofaresearchproject.(Powers&Knapp,2011).

theoverallmethodoftheresearchstudy. Shouldflowfromthestatedpurpose Shouldprovideaplantoanswertheresearch

questionsandtestthestatedhypothesis(Schmidt&Brown,2012).


4/72

ReviewWhatisaVariable?Variabletheoperationalizationofaconcept

Classifyingvariables/Variabletypes Independent(IV)Thevariablethatisthoughttobethe

cause

Inexperimentalresearch,theresearchermanipulatesthisvariabletoseehowthateffectstheresults

Dependent(DV)Theoutcomethatisbeingstudied.Thisisthevariablethatisofprincipleinterestinthestudy


5/72

MoreonVariables Othervariabletypes

InterveningAvariablethatisbetweenanIVandaDVinacausalsequenceandsoeffectsthecausalrelationship

X Z Y

ExtraneousVariableApotentiallyconfoundingvariablethatisnotofinteresttothestudy Extraneousvariablesmustbecontrolledbecausetheycan

confuseorconfoundtherelationshipbetweentheindependentanddependentvariable Failuretocontrolextraneousvariablesleadstobiaswhichresultsin

invalidresults


6/72

Quan&ta&veDesigns:2Broad

CategoriesNonexperimentalDesigns ExperimentalDesigns

Describemeasurablephenomenon

Explainrelationshipsamongvariables

Describedifferencesamongvariables

Predictingrelationshipsanddifferencesamongvariables

Examinecausality

DoesXcauseY?


7/72

Quan&ta&veDesign:Important

ConceptsCausality Probability

Definition:Therelationshipthatexistsbetweenacauseandits

effect(Schmidt&Brown2012). 3conditionsforestablishing

causality: Theremustbeatemporal

relationship Example:XmustprecedeY

intime

Therelationshipbetweenvariablesmustbestrong

NoothervariablesorconditionscanaccountfortherelationshipbetweenX&Y

Thelikelihoodorchancethataneventwilloccurinagivensituation(Schmidt&Brown,2012)

Inresearchonhumans,causalityisnevercompletely

certain Ideaofprobabilityor

likelihoodofacauseeffectrelationshipismorecommoninnursingresearch


8/72

Cause,Effect,ProbabilityItsnot

thatsimple! TheoreticalCauseand

EffectRelationship:

X Y

Reality:Anyeffect(Y)istheresultofmultiplecontributingcauses


9/72

Quan&ta&veDesign:Important

ConceptsControl Manipulation

Theresearchersattempttoclarifytherelationship

betweenanindependentandadependentvariablebyeliminatingotherthings(variables)thatmightaffecttherelationship

Eliminatingoutsideinfluencesbyclosecontrolofconditions

Usingstatisticstoeliminatetheeffectsofothervariables

Control/interventionoftheindependentvariablebytheresearcher

Definingcharacteristicofanexperimentalstudy

Donebycreatingdifferentexperimentalconditions Examples

Dividingsubjectsintotwoormoregroupssubjectedtodifferentconditions


10/72

ControllingBiasinResearch

Studies Biasresultswhenextraneousvariablesinfluencethe

relationshipbetweentheIVandtheDV

Controllingforbias: RandomizationSelection,assignmentorarrangement

ofelementsbychanceinanattempttoeliminateoramelioratetheeffectsofextraneous

Twomajormethods RandomSampling RandomAssignment


11/72

MethodsofRandomiza&onRandomSampling RandomAssignment

Methodsthatassurethateverypersoninthe

populationhasanequalchanceofbeinginthesample Thisassuresthatsampleis

similartothepopulationitis

takenfrom Examplesofrandomization

Coins Tableofrandomnumbers Computergeneratedlists

Methodsthatassurethateverysubjectinthestudyhasanequalchancetobeineithertheexperimentalorthecontrolgroup

Increasesthelikelihoodthatextraneousvariableareevenlydistributedbetweenthetwogroupsanddonotdisproportionatelyaffectoneofthegroups


12/72

TwomoreimportanttermsBetweenGroups WithinGroups

Researchstudycomparesthetwogroupsofsubjectson

somevariable(s). Examples:

Experimentalstudies Studiescomparingtwopre-

existinggroupsonacertain

variable BMIofpersonswithType2

DiabetesversuspersonswithoutDiabetes

Incidenceoflungcanceramongsmokersversusnon-smokers

Comparesagroupwithitselfonsomevariable(s)atmorethan

onepointintime. Oftenthegroupismeasuredpre

andpostintervention Examples:

HemoglobinA1CofindividualswithType2Diabetesbeforeandafter3monthsofMetformin

Self-esteemofteengirlsbeforeandafterparticipatinginatargetedafter-schoolintervention


13/72

StudyValidityValidity:Theextenttowhichtheresultsofthestudy

aretrue,logical,reasonable&justifiablebasedontheevidencepresentedinthestudyAccuracyofthestudyfindingsValidityisensuredbytheresearcherthroughcareful

methodswhichcontrolforextraneousvariables Carefulselectionofsubjects Meticulousattentiontostudyprocedures/protocols Accuratemeasurementofstudyvariables Correctstatisticalmethods Accurateinterpretationofresults


14/72

4TypesofStudyValidity(SchmidtBrown,2012)*

*

*


15/72

ThreatstoValidityAthreat(tovalidity)isanythingthatweakensthe

internalorexternalvalidityofastudy

Nursesneedtobeabletoevaluatestudiestodetermineiftheresultsaretrue.(noorminimalthreatstostudyvalidity)


16/72

History Somethingoutsidethestudychangesandthischanges

results(dependentvariable)

Thisproblemismostlikelytoaffectsresearchthattakesplaceovertime

Controlgroupisbestdefense

Testing Measurementitselfaffectsresults

Repeatedtestingcausesfamiliaritywithtest Useofpretestaffectspost-testresults

16

WhatthreatensInternalValidity?


17/72

Instrumentation Achangeinthewayavariableismeasuredduringthecourseofa

studycanaffectvalidity Timingofmeasurements Howthevariableismeasured(typeofinstrument) Changeindatacollectors

Subjectmortality Lossofsubjectsduetoanyfactorrelatedtothedependentvariable

Death Studywithdrawal Losstofollow-up

Selectionbias Subjectsinexperimentalandcontrolgroupsarenotsimilar

enough Thisproblemaffectsstudieswithoutrandomselectionthemost

17

WhatthreatensInternalValidity?


18/72

WhatthreatensInternalValidity? Maturation

Subjectschangeduetothepassageoftimeratherthanduetotheexperimentalintervention Aging Growth Diseaseprogression

Thisproblemismostlikelytoaffectsresearchthattakesplaceovertime

StatisticalConclusionValidity TypeIIerrors:Researchersconcludethatthereisno

differenceorrelationship,whentheretrulyisone Thisisariskwithsmallsamplesizes

Unreliablemeasurementinstruments


19/72

Reactivity HawthorneEffect

Subjectsbehaviordifferentlybecausetheyknowtheyareinastudy

Thisisusuallyunconsciousontheirpart ExperimenterEffect

Somecharacteristicoftheresearcherordatacollectoraffectstheresults

Example:Thepersonadministeringtheinterventionisprettyandnicesothesubjectstrytopleaseher

Double-blindstudycanhelpamelioratethiseffect19

WhatthreatensExternalValidity?


20/72

ConstructValidity Instrumentsusedtomeasuretraitsmustbeevaluated

tomakesurethattheyaremeasuringthattraitalone

Example:Measuresofanxietymightoverlapwithmeasuresofdepression

NoveltyEffect(notinyourbook) Subjectsresponddifferentlybecausetheyknowthe

interventionortreatmentisnewordifferent

Itisthenoveltynottheactualinterventionthatiscausingchange

20

WhatthreatensExternalValidity?


21/72

WhatthreatensExternalValidity? SelectionEffects

Samplesthatdonotrepresenttheappropriatepopulationlimitexternalvalidity Thisisminimizedbytheresearcherselectinganappropriatesamplefor

thetargetpopulation,usingrandomselectionofsubjectsandrandomassignmenttogroups

InteractionEffects Treatmentandsubjectselection

TheIVmaynotaffectallindividualsinthewayitdoesinthestudy.Thismaybeduetodifferencesbetweenthestudysampleandother

groups Treatmentandsetting

Theresultsofthestudymayberelatedtothespecificsettingandmaynotbeapplicabletoothersettings

Treatmentandhistory Theresultsofthestudymaybeaffectedbyeventsoccurringatthesame

timeasthestudybutoutsideofthestudy


22/72

22

ValidityParadox

InternalValidity ExternalValidityInternalValidity ExternalValidity


23/72


24/72


25/72

FundamentalsofDesign:Review:

WhatisResearchDesign? ResearchDesignistheoverallplanforobtaining

answerstotheresearchquestionsorfortestingtheresearchhypotheses.

Spellsoutbasicstrategiestoinsurethatresultsareaccurate,interpretableanduseful Sample/sampling Datacollectionandmanagement Plannedstatisticalanalysis Measurestakentomaintainrigorandcontrolthreatsto

validity

25


26/72

MoreReview Theresearchdesignmustfitthestudypurpose,

questionand/orhypothesis

26


27/72

HowareResearchDesigns

Classified? ByTimeElement

Retrospective Longitudinal/Prospective Cross-Sectional

ByType Nonexperimental Experimental

Quasi-Experimental

ByFunction Theresearchdesignmustfitthestudypurpose,question

and/orhypothesis Whatistheresearchertryingtodo?

27


28/72

Classifica&onbyTimeElement Retrospective/ExPostFactoDesigns Datacollectedaboutpastevents Maybemorethanonedatapoint

Looksatwhathappenedtounderstandacurrentphenomenon Longitudinal/ProspectiveDesigns

Datacollectedatmorethanonepointintimebeginningatthestartofthestudy

RepeatedMeasuressameinstrumentsusedtocollectdataaallmeasurementpointsallowingforcomparisonovertime.

Allowsresearchertoexaminethingsinrealtime Cross-Sectional

Datacollectedallatonepointintime Cannotbeexperimental Theoryimportanttotheinterpretationofthesestudies WhatshappeningNOW!

Doesnotgiveanyinformationaboutchangesovertime28


29/72

MoreonDesignsXTimeElements Retrospective/ExPostFactoDesigns

Case-ControlStudies Usedataonpatientswithaconditionthatiscollectedbypatienthistoryto

determineacause HIV/AIDS,Smoking,Type2Diabetes,Hypertension

Longitudinal/Prospective CohortStudiesfollowsgroup(s)forwardovertimeandlearnswhat

conditionsdevelopinthegroups WomenwhotakeHRTvswomenwhodont Personswithdiabeteswhosesugaristightlycontrolledversusthosewhose

sugarisnot

CrossoverstudySamegroupofsubjectsreceivemorethanonetreatmentovertimeorserveastheirowncontrolgroups

Cross-SectionalDesigns CohortComparisonStudieslookatmorethanonegroupofsubjectsat

thesamepointintime Sexualbehaviorinteenswhogotopublicschoolversusreligiousschools DepressioninwomenwhohadaTOPversusthosewhohavenot Weightinstudentswhoeatinthedininghallsversusthosewhocookfor

themselves

29


30/72

Nonexperimental(Descriptive) Studiesasis Nomanipulationofvariables Canbequalitativeorquantitative

Experimental&Quasi-experimental Studywhatwouldhappenif Involvesmanipulationofvariable(s)Alwaysquantitative ControlimportantAlwaysprospective/longitudinal

30

Classifica&onbyType


31/72

NonexperimentalDesigns Functionisdescription Helpusunderstandthingsastheyarecurrently

Describephenomenon

Explainrelationshipsanddifferences Predictrelationshipsanddifferences Donotclaimtoindicatecauseandeffect

Nomanipulationofindependentvariable Researchcontrolstillimportant Canbequantitativeorqualitative

31


32/72


33/72


34/72

SamplePathModel:Psychological

AdjustmenttoBreastCancerDiagnosis

34

Epping-Jordan,J.etal.(1999).PsychologicaladjustmenttobreastCancer:Process

ofemotionaldistress.HealthPsychology,18(4),315-326.


35/72

ExperimentalDesigns 3EssentialElementsofTrueExperimentalDesign

1. Randomization Randomselection Randomassignment

2. Control Extraneousvariablesarecontrolled

3. ManipulationoftheIV

35


36/72

Attemptsbyresearchtocontrolconditionsinordertodecreasevariability(change)intheD.V.thatiscausedbyeventsorfactorsindependentofthestudydesign

ExtrinsicFactors relatedtoenvironment/researchsituation

IntrinsicFactors relatedtocharacteristicsofsubjectsthemselves

36

WhatisResearchControl?


37/72

AreasofResearchControl Sampling

Inclusion&exclusioncriteria Randomsampling Randomassignmenttogroups

ResearchConditions Sameforallsubjects

DataCollection Carefulchoiceofreliableandvalidinstruments Useofcomparisonorcontrolgroups Uniformityofdatacollectionagentsandconditions Datacollectorsarealltrainedtodothesamethings

DataManagement

4/4/13 37

Whatiselementsmustbe

Controlled?


38/72

6TypesofTrueExperimental

Design1. 2-group,pretest-posttest2. 2-group,posttestonly3. Solomon4-groupdesign

2groupsgetinterventionandtwoarecontrol Seenextslide

4. Multipleexperimentalgroups 2experimentalgroupswithonecontrolgroup

5. Factorial Tests>1interventioncanbetestedatsametime

6. CrossoverDesign 2treatmentsandsubjectsreceivebothtreatmentsin

randomorder


39/72

CommonExperimentalDesigns


40/72


41/72

Quasi-ExperimentalDesigns Donotmeetalltheessentialcomponentsof

experimentaldesign ManipulationoftheIVisalwaysincluded Maynotinclude:

Randomization Someelementsofresearchcontrolmaybemissing

3CommonQuasi-ExperimentalDesigns Nonequivalentcontrolgrouppretest-posttest Timeseriesdesigns Preexperimentaldesigns


42/72


43/72

CommonQuasi-ExperimentalDesigns

43


44/72

4/4/13 44

ReviewofClassifica&onofQuan&ta&veDesign


45/72


46/72


47/72

Objec&ves:SamplesandSampling Evaluatearesearchreportforappropriatenessof

sampleandsamplingmethods

Evaluateresearchforadequacyofsamplesize Utilizeprinciplesofevidence-basednursingto

determineiftheresearchfindingsareapplicabletoaclinicalproblembasedonsamplecomposition

4/4/13 47


48/72

Whatisapopula&on? Thetotalnumberofpersonsinhabitingacountry,city,oranydistrictorarea.(www.dictionary.com)

Inresearchterms:Anentiresetof

people,orhospitalsorwhateverisofparticularinteresttoaresearcher(Powers&Knapp,2011)

Sometimescalledthestudypopulationorthetargetpopulation

4/4/13 48


49/72


50/72

Study/TargetPopula&on Definedbyasetof

specificcharacteristicsthattheyhavein

common

Canyouthinkofsomeexamplesofpopulationsfromtheresearcharticlesyouhavereadforthisclass?

4/4/13 50


51/72

AccessiblePopula&on Termusedtoindicatethepartofthepopulationthat

aresearcherhasaccessto

Example: Studypopulation:Allmenoverage50with

hypertension

Accessiblepopulation:Menoverage50withhypertensionwhoattendalargeclinicinacertaincity

4/4/13 51


52/72

StudyingPopula&ons

Aresearcherwantstostudythegrowthanddevelopmentofveryprematureinfantsduringtheirfirst5yearsoflife.

5,000veryprematureinfantsareborninherstateeveryyear.

Whatdoyouthinkshedoes?

4/4/13 52


53/72

Howissamplinglikemakingsoup?

4/4/13 53


54/72

Whatisasample?

Asubsetofapopulationthatisstudiedtolearnsomethingaboutthepopulation

Ideallythesamplehasthesamespecificcharacteristicsasthepopulation

AsamplethatmeetsthiscriterioniscalledREPRESENTATIVE

4/4/13 54

Inthisillustration,whichcirclerepresentsthepopulationand

whichrepresentsaofthepopulation?


55/72


56/72


57/72


58/72

Samplingtechniquesareusedto: SelectasamplethatisaslikethepopulationaspossibleAvoidsamplingbias Controlforextraneousvariables

extraneousvariables (Review:extraneousvariablesarethosethatarenotbeing

measuredinthestudybutthatmightaffectstudyresults).

Example:AresearcherwantstomeasuretheeffectofaneducationalprogramrelatedtosugarintakeonteensBMIandbloodsugarlevels

Examplesofextraneousvariables:1. Unknowntoher,abeveragedistributorbeginsgivingout

freesweetenedicedteaduringafter-schoolactivities.

2. SomeoftheteenshavefamilymemberswithType2diabetesandshedoesnotknowthis.

4/4/13 58


59/72

GoalofSampling QuantitativeResearch

Todecreasevariabilityamongthesubjects. Subjectsshouldbeasalikeaspossibletoavoidpre-

existingdifferencesthatmightaffectresults QualitativeResearch

Triestoincreasevariabilityamongthesubjectstolearnasmuchaspossibleaboutatopic.

Trytogetsubjectswhorepresentasmanydifferentaspectsofthesubjectunderstudyinordertogetallpossibleviews. PurposiveorTheoreticalSampling

4/4/13 59


60/72

SamplingDecisionsareCri&calin

Quan&ta&veResearch Oncetarget/studypopulationisknown,the

researchertriestochoosethesamplethatisbestforthestudy.

Mostlikeorrepresentativeofthepopulation Samplingcriteriaareusedtoselectsamplethatisas

representativeaspossible

4/4/13 60


61/72


62/72


63/72

ProbabilitySampling Randomsamplingshouldresultinasamplelikethe

studypopulation

Characteristics(age,gender,diseasestatus)distributedequallyamongthegroups

4/4/13 63


64/72

Quan&ta&veSampling:Probability

Sampling SimpleRandom samplesarecomputer-generatedortableofrandomnumbers

orsimilar StratifiedRandom

Subjectsaredividedbysomecharacteristic(e.g.age,clinic,race)andthenrandomsamplingisdonewithinthestrata

ClusterSampling Randomselectionofsubjectsfromwithinknownclustersof

individualswiththecharacteristic Example:subjectswithprostatecancerfrom6largecancercenters.Each

clinicgroupisacluster.Thesubjectsarethenrandomizedwithineachcluster.

SystematicSampling Usesalistofallmembersofanaccessiblepopulation.

PopulationsizedividedbysamplesizeandtheneveryKthindividualselectedfromthelist.(Nottruerandomsampleasnotallhaveequalchanceofinclusion)

4/4/13 64


65/72


66/72

SampleSizeinQuan&ta&ve


67/72

p Q

Research SizeMatters!

Largersamplesaremorelikelytoberepresentativeof(like)populationofinterest

Largersamplesaremorelikelytoresultinstatistical

significance. Resultsusingsmallersamples

morelikelytobetheresultofchance.

PowerAnalysisAstatisticalprocedurethat

allowstheresearchertocomputetheminimum#neededinsampletodetectarealdifferenceorrelationship Lookforthiswhenreading

quantitativeresearch.4/4/13 67


68/72


69/72

WhataboutMortality SubjectMortality=lossofsubjectsduringthe

courseofastudyforanyreason

Whatdoyouthinkcancausesubjectmortality?

4/4/13 69


70/72

MortalityandSubjectBiasAttritionmayindicatetheexistenceofsamplingbias.

Self-selection Failuretocomplete

surveyorstudy

Researchershoulddiscussattritioninreportingstudyand

explainpotentialreasonsforit.

4/4/13 70


71/72

Evalua&ngQuan&ta&veSamples

forEBN1. Wasthereanidentifiablesectiononsamplingintheresearchreport/article?

2. Wasenoughinformationgiventounderstandhowandwhysamplechosen?

3. Isthereenoughinformationgivenaboutthesampletotelliftheresearchappliestoyourclinicalpopulation?

4. Forquantitativeresearch: Isthereinformationthattellsmethatthesampleis

largeenough.

4/4/13 71


72/72

Documents

Understanding Quantitative Research + Sampling