View
218
Download
0
Embed Size (px)
Citation:Neuhauser,C.HypothesisTesting.Created:November29,2009Revisions:Copyright:2009Neuhauser.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommonsAttributionNonCommercialShareAlikeLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,andallowsotherstotranslate,makeremixes,andproducenewstoriesbasedonthiswork,providedtheoriginalauthorandsourcearecreditedandthenewworkwillcarrythesamelicense.Funding:ThisworkwaspartiallysupportedbyaHHMIProfessorsgrantfromtheHowardHughesMedicalInstitute. Page1
HypothesisTesting
LearningObjectives
Aftercompletingthismodule,thestudentwillbeableto
carryoutastatisticaltestofsignificance calculatetheacceptanceandrejectionregion calculateandinterpretthepvalueofastatistical
test
calculateandinterprettype1andtype2errors calculatethepowerofatest
KnowledgeandSkills
Concepts:nullhypothesis,alternative,teststatistic,rejectionregion,acceptanceregion,pvalue,significancelevel,type1error,type2error,falsepositive,falsenegative,powerofatest
Resamplingmethod Fishersexacttest
Prerequisites
binomialdistribution hypergeometricdistribution Normaldistribution Sampleaverage Samplestandarddeviation macrosinExcel
Citation:Neuhauser,C.HypothesisTesting.Created:November29,2009Revisions:Copyright:2009Neuhauser.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommonsAttributionNonCommercialShareAlikeLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,andallowsotherstotranslate,makeremixes,andproducenewstoriesbasedonthiswork,providedtheoriginalauthorandsourcearecreditedandthenewworkwillcarrythesamelicense.Funding:ThisworkwaspartiallysupportedbyaHHMIProfessorsgrantfromtheHowardHughesMedicalInstitute. Page2
PrologueTheproblemofdecisionmakingisubiquitous.Almostdaily,youcanreadinthenewsaboutstudiesthatleadtorecommendationsbasedonstatisticalevidence.TheU.S.DepartmentofHealthandHumanServicesAgencyforHealthcareResearchandQuality(http://www.ahrq.gov/)provideshealthcarerecommendations,forinstance,throughitsU.S.PreventiveServicesTaskForce(http://www.ahrq.gov/clinic/uspstfix.htm),anindependentpanelofexpertsinprimarycareandprevention,whichreviewsresearchresultsanddevelopsrecommendations.Theserecommendationsarebasedonanalysesoftensorhundredsofclinicalstudies,andrecommendationsmaychangeasnewevidenceaccumulatesovertime.
Frequently,clinicalstudiesarephrasedintermsofhypothesistesting.Forinstance,ifanewtreatmentforadiseaseisdeveloped,wemightwishtoknowwhetheritperformsbetterthanthecurrenttreatment.Wesetupaclinicaltrialwherepatientsarerandomlyassignedtooneortheothertreatment.Wethencomparethenumberofsuccessfultreatmentsineachgroup.Letsassumethatthetwogroupshavethesamenumberofpatients.Inordertoconcludethatthenewtreatmentisbetterthanthecurrenttreatment,wewouldneedtodemonstratethatthenumberofsuccessfultreatmentsinthenewtreatmentgroupislargerthanthenumberofsuccessfultreatmentsinthecurrenttreatmentgroup.Thequestionishowmuchlargerthenumberofsuccessfultreatmentsinthenewtreatmentgroupwouldneedtobetoconvinceotherinvestigatorsthatthenewtreatmentisindeedbetter.Thesekindsofquestionscanbeansweredwithintheframeworkofhypothesistesting.
InclassActivity1
Assumethecurrenttreatmentforadiseaseissuccessfulin30%ofallcases.Anewtreatmentisbeingdevelopedandapreliminaryclinicaltrialshowedthat5outof10patientsweresuccessfullytreated.Canyouconcludethatthenewtreatmentismoresuccessful?
Ifthenewtreatmentwasnotbetterthanthecurrenttreatment,wewouldhypothesizethatthenewtreatmenthasprobability0.3ofbeingsuccessful.Alternatively,ifthenewtreatmentisbetterthanthecurrenttreatment,wewouldhypothesizethatthenewtreatmenthasprobabilitygreaterthan0.3ofbeingsuccessful.
Ifthenewtreatmenthasthesamelikelihoodofsuccessthanthecurrenttreatment,namelyprobability0.3,thenthenumberofpatientsinthesmallclinicaltrialwhoaretreatedsuccessfullyunderthenewtreatmentisbinomiallydistributedwith10trialsandsuccessprobability0.3.ThefollowingtablewascreatedinEXCELusingtheBINOMDISTfunctionandshowsthisprobabilitydistribution:
Citation:Neuhauser,C.HypothesisTesting.Created:November29,2009Revisions:Copyright:2009Neuhauser.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommonsAttributionNonCommercialShareAlikeLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,andallowsotherstotranslate,makeremixes,andproducenewstoriesbasedonthiswork,providedtheoriginalauthorandsourcearecreditedandthenewworkwillcarrythesamelicense.Funding:ThisworkwaspartiallysupportedbyaHHMIProfessorsgrantfromtheHowardHughesMedicalInstitute. Page3
x P(X=x)
0 0.02821 0.12112 0.23353 0.26684 0.20015 0.10296 0.03687 0.00908 0.00149 0.0001
10 0.0000
Weseethattheprobabilityoffiveormoresuccesseswhenthesuccessprobabilityis0.3is
0.1029 0.0368 0.0090 0.0014 0.0000 0.1502+ + + + =
Thus,itisnotunlikelytosee5(ormore)outof10patientsrecoverwhenthesuccessprobabilityofrecoveryis0.3.Weconcludethatthereisnotenoughevidencetoconcludethatthenewtreatmentisbetter.
Discussinyourgroupthefollowingquestions:
1. Whydidweadduptheprobabilitiesintheaboveexample?2. Wouldyoubeabletoconcludedefinitivelyfromthisstudythatthenewtreatmentisntanybetter?3. Whatwouldbeyournextstepindeterminingwhetherthenewtreatmentisbetter?
Citation:Neuhauser,C.HypothesisTesting.Created:November29,2009Revisions:Copyright:2009Neuhauser.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommonsAttributionNonCommercialShareAlikeLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,andallowsotherstotranslate,makeremixes,andproducenewstoriesbasedonthiswork,providedtheoriginalauthorandsourcearecreditedandthenewworkwillcarrythesamelicense.Funding:ThisworkwaspartiallysupportedbyaHHMIProfessorsgrantfromtheHowardHughesMedicalInstitute. Page4
InclassActivity2
Supposeyouhaveacoininyourpocket.Youwanttodecidewhetherthecoinisfairorbiased.Youhypothesizethatthecoinisfair.Totestthishypothesis,youtossthecoin30times.Thenumberofheadsisbinomiallydistributedwiththenumberoftrialsbeing30andtheprobabilityofheads(success)being0.5.Belowisthehistogramoftheprobabilitydistribution.
Supposetheexperimentresultedin18headsand12tails.Discussthefollowingquestionsinyourgroup:
1. Whatcanyousayaboutthecoin?Isitafaircoinorabiasedcoin?2. Whatwouldyourconclusionbeiftheexperimentresultedin24headsand6tails?3. Whatcriteriadidyouusetomakethedecisionineachofthetwocases?4. Canyoubesurethatyourdecisioniscorrect?
Citation:Neuhauser,C.HypothesisTesting.Created:November29,2009Revisions:Copyright:2009Neuhauser.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommonsAttributionNonCommercialShareAlikeLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,andallowsotherstotranslate,makeremixes,andproducenewstoriesbasedonthiswork,providedtheoriginalauthorandsourcearecreditedandthenewworkwillcarrythesamelicense.Funding:ThisworkwaspartiallysupportedbyaHHMIProfessorsgrantfromtheHowardHughesMedicalInstitute. Page5
SomeTheoryInbothInclassActivities,youhadtomakeadecisionbetweentwoalternatives.Inthefirstcase,youneededtodecidewhetherthenewtreatmentwasbetterthanthecurrenttreatment.Inthesecondcase,youneededtodecidewhetherthecoinwasfairorbiased.Inbothcases,youreliedonaprobabilitymodel,andyoubasedyourdecisiononhowlikelytheoutcomeoftheexperimentwascomparedtotheexpectationofthemodel.Inbothcases,therewasalsothepossibilitythatyouarrivedatthewrongdecision.
Inthefollowing,wewilldiscussthebasicelementsofhypothesistesting.Wewillusetheexampleofthefaircoinversusthebiasedcoinbecauseofitssimplicity.Onehypothesisisthatthecoinisfair,thatis,thattheprobabilityofheadsis0.5.Thealternativehypothesisisthatthecoinisbiased,thatis,theprobabilityofheadsisdifferentfrom0.5.Webaseourdecisionofwhetherornotthecoinisfaironcomparingtheresultofourexperimenttowhatweexpectbasedonaprobabilisticmodel.Namely,ifthefractionofheadsintheexperimentiscloseto1/2,theexperimentprovidesevidenceforthecoinbeingfair;ifthefractionofheadsiseitherloworhigh,theexperimentprovidesevidenceforthecoinbeingbiased.
Thehypothesisthecoinisfairiscalledthenullhypothesisandisdenotedby 0H .Thealternativethe
coinisbiasedisdenotedby 1H .(Wewillsaymoreaboutwhichofthetwohypothesesisthenull
hypothesisandwhichisthealternativelater.)Wesummarizethisas
=
0
1
: 0.5
: 0.5
H p
H p
Wedesignedanexperimentinwhichwetossedthecointhirtytimes.Thedatacollectedintheexperimentprovidedevidencefororagainstthenullhypothesis.Thedatainourexperimentwerethesequenceofheadsandtailsinthethirtytrials.Thedatasuggestthatwecancalculateasinglenumber,namelythenumberofheads,whichwecancompareagainstwhatwewouldexpectunderthenullhypothesis.Thissinglenumberiscalledtheteststatistic.Aprobabilisticmodelfortossingafaircoinallowsustocalculatetheprobabilitydistributionoftheteststatistic.Namely,underthenullhypothesis,thenumberofheadsisbinomiallydistributedwith30trialsandsuccessprobability = 0.5p .Inthe
experiment,weobserved18heads.Howlikelyisitthatweobserve18ormoreheads?IfXdenotesthe
numberofheads,weareaskingfor ( 18)P X ,whichcanbecalculatedbyaddinguptheprobabilitiesof
theevents{ } { } { }= = =18 , 19 ,... 30X X X .Refertothespreadsheet(tabFairCoin)toverifythat
=( 18) 0.1808P X
Citation:Neuhauser,C.HypothesisTesting.Created:November29,2009Revisions:Copyright:2009Neuhauser.ThisisanopenaccessarticledistributedunderthetermsoftheCreativeCommonsAttributionNonCommercialShareAlikeLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,andallowsotherstotranslate,makeremixes,andproducenewstoriesbasedonthiswork,providedtheoriginalauthorandsourcearecreditedandthenewworkwillcarrythesamelicense.Funding:ThisworkwaspartiallysupportedbyaHHMIProfessorsgrantfromtheHowardHughesMedicalInstitute. Page6
Sincethealternativeistwosided
