Upload
nadakhan
View
216
Download
0
Embed Size (px)
Citation preview
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
1/30
DrGillianLancaster
PostgraduateStatisticsCentreLancasterUniversity
TrialsinPrimaryCare:design,conductandevaluationofcomplexinterventions
CentreforExcellenceinTeachingandLearning
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
2/30
RSSPrimaryHealthCareStudyGroupCoauthorsofSMMRpaper:
MikeCampbell,Sheffield
SandraEldridge,QueenMaryLondon
AmandaFarrin,LeedsMauriceMarchant,EastSussexPCT
SaraMuller,Keele
RafaelPerera,OxfordTimPeters,Bristol
TobyPrevost,KingsCollege
GretaRait,UCL
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
3/30
1.Introduction
ResearchinPrimaryCareistimeconsumingandoftenchallenging
Itrequiresextensiveplanning&prep
Interventionsareoftencomplexand
presentarangeofproblemseg.
Workinginhealthcaresetting
Sensitivitytolocalcontext
Logisticsofapplyingexperimentalmethods
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
4/30
Whatmakesanintervention
complex?
Interactionsbetweencomponentsinexperimentalandcontrolarms
Difficultyofbehavioursrequiredbythosedeliveringorreceivingtheintervention
Organisationallevelstargetedbytheintervention
Variabilityofoutcomes
Degreeofflexibility/tailoringofinterventionpermitted
Willitworkineverydaypractice?
NB.takenfromMRCguidelines
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
5/30
Guidance
MRC
documentDevelopingandEvaluatingComplexInterventions
www.mrc.ac.uk/complexinterventionsguidance
CraigP.etal.BMJ2008,337:a1655
BMJpaper(CampbellNCetal.2007,334:4559)DesigningandEvaluatingComplexInterventionsto
improvehealthcare
Casestudies
http://www.mrc.ac.uk/complexinterventionsguidancehttp://www.mrc.ac.uk/complexinterventionsguidance8/6/2019 Oxford Talk SMMR Paper July 2011 Final
6/30
KeystatisticaldesignissuesIPhases
given in MRCguidanceframework
Key elements in
designing andevaluating complexinterventions
General points to consider Key statistical design
issues addressed in ourpaper
Development Background andcontext(For more informationand examples see MRC
and Campbell et al.)
Socio-economic background;Underlying cultural
assumptions;Health service system;
Government initiatives;Preventative policies
Defining andunderstanding theproblem(See above docs)
Prevalence of condition;Population most affected;How condition is
caused/sustained;Potential for intervention and
improvementConceptualising theproblem(See above docs)
Levels of complexity of healthproblem and co-morbidity;
Risk factors and factorsinfluencing changes overtime;
Patient beliefs, symptoms andadherence to treatment
Gathering evidence Systematic reviews;Epidemiological research;Qualitative research;Expert opinion
Using evidence from primarystudies, systematic reviewsand qualitative studies toinform study design
Developing theintervention
Identify key processes andmechanisms for delivery;
Potential beneficial effect;
Define target group;Optimise best treatment
combinations
Conducting primary careresearch in the UK:complying with research
governance and assessingquality of care using theQuality and OutcomesFramework
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
7/30
KeystatisticaldesignissuesII
Phases given inMRC guidanceframework
Key elements indesigning andevaluating complexinterventions
General points to consider Key statistical designissues addressed in ourpaper
Evaluation Developing and
optimising trialparameters
Testing the feasibility and
integrity of the trial protocol;Consideration of appropriate
primary/secondaryendpoints;
Recruitment and retentionstrategies;
Method of randomisation to
minimise imbalance;Sample size considerations
Pilot studies and pre-trial
modelling;Selection of outcome
measures for effectivenessand quality;
Recruitment of practices andparticipants;
Choosing the method of
randomisation;Sample size and between
trial variationData collection andanalysis
Data collection forms;Design of database;Monitoring procedures;Awareness of issues of data
analysis for different studydesigns
Choosing the method of
analysis: cluster specificversus marginal modelsImplementation Getting evidence into
practice(See new MRCguidance document)
Publication and disseminationstrategy;
Stakeholder involvement;Benefits, harms, costs for
decision making;Recommendations
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
8/30
2.Usingevidencefromprimary
studies,systematicreviewsand
qualitativestudiesinthedesign
Muchhighqualityresearchlacksgeneralisability (externalvalidity)
Interventionmaynotbeeasilyimplementedinpractice(Who?How?Duration?)
Strongargumentforcarryingoutresearchin
themostappropriatecontextandsettingEg.Canwetrustestimateofeffectsizewhen
interventionstudiestolowerBPafterstroke
aremostlycarriedoutinsecondarycare?(Mant etalBMJ2006)
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
9/30
Usefulbecausebasedonclearlyformulatedresearchquestionsandmethodology
Quality
of
included
papers
has
been
appraised Summary(pooled)estimateofeffectsize
Feasibility,acceptabilityanduptakeof
intervention
can
be
measured
by
level
of
attrition
ofparticipants
Eg.RelativeattritionhasbeenusedtocomparelevelsofattritionacrossoralanticoagulationandDiabetestypeIItrials(Hennekens etal.BMCRes.Methods2007)
Systematicreviewsofdiagnostictestandmethodcomparisonstudiesalsousefulforselectinganappropriatemeasurementmethod ortechnique
SystematicreviewsofRCTs
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
10/30
Especiallyusefulwhenplanningorevaluatingacomplexintervention
Canbeused:
Before thetrialtoexploreissuesofdesigneg.barrierstorecruitment;acceptabilityof therandomisationfromapatientsperspective
During thetrialtounderstandandunpacktheprocessesofimplementation andchange
After thetrialtoexplorereasonsforthefindingseg.arefindingsinlinewithunderlyingtheory;acceptabilitytodeliverersandreceivers;comparisonswithpatientreportedoutcomes;
thevalueoftheintervention asanevaluativeassessmentandtoaidinterpretation
Qualitativestudies
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
11/30
3.
Conducting
primary
care
researchintheUK
Publicgenerallytrustsacademicresearch Researchgovernance ensuresresearchintegrity
toupholdthepublicsconfidence,toprotect
participantsfromabuse,andtoprotectresearchersfromaccusationsofmisconduct
Widerangeoflegalrequirementseg.
European
Clinical
Trials
Directive DataProtectionAct
EthicalapprovalNB.muchdebateaboutwhetherRECsshould
examinestatisticalissues&methodologicalrigour
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
12/30
GPsareusuallyselfemployedorworkwithina
limitedcompany;contracttoNHS
NHSPrimaryCareTrusts(PCTs)commissionGPs
serviceswithintheirgivenarea
PCTsfacilitateresearchlocallytoensure
researchintegrity;researchreviewcommittees
PrimarycareresearchofteninvolvesseveralGP
centresacrossmultiplePCTs
verytimeconsumingtoobtainapproval;
honorarycontracts;CRBchecksetc.(eg.6months)
NIHRhaverecentlyintroducedguidanceanda
ResearchPassportSystemtohelptheprocess
Researchgovernance
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
13/30
Tomonitorqualityofcareofpatients,financial
incentives(upto30%ofGPincome)havebeen
introducedthroughtheQualityandOutcomes
Framework(QOF)
QOFhas5domainsofincentivisation
oClinicalcare
oOrganisation
oPatientexperience
o
Education
and
trainingoOtheradditionalservices
Pointsareawardedaccordingtotheworkload
neededtoachievetargetsandprevalenceofdisease(age,sex,deprivation)inthearea
ResearchpotentialofQOF
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
14/30
TouseQOFindicatorsinresearcheg.toassess
differencesinqualityofcare,therearecertain
problemstoovercome:
o Exclusionseg.failuretoattendforassessment,
frailtyofcondition,refusetreatment
o DifferencesbetweenGPPracticeseg.how
conditionsarerecorded,howinterventionsare
assessed,compositionandskillsofpracticestaff
QOFisprimarilypaymentdrivenandnotcreated
forresearchpurposes
Researchdatabaseshavebeencreatedeg.GPRD,
Qresearch,usingsamplesofGPpractices
ResearchpotentialofQOF
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
15/30
4.Useofpilotstudies
Important
pre
requisite
for
funding Oftenadhocsmallstandalonestudies
Subjecttopublicationbias
Isthereadifferencebetweenafeasibility
andapilotstudy?
Pilotstudiesaddresstheintegrityofthestudyprotocol
Needclearlistofkeyobjectives
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
16/30
Keyobjectivesofapilotstudy
Testintegrityofstudyprotocol
Samplesizecalculation
Recruitmentandconsentrates Developandtestimplementationanddelivery
oftheintervention
Acceptabilityoftheintervention Trainstaffindeliveryandassessment
Selectionofmostappropriateoutcome
measures
(endpoints) Randomisationprocedure
Pilotdatacollectionforms/questionnaires
Prepareandplandatacollectionandmonitoring
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
17/30
Example UKBEAMtrial
UKBackPain,Exercise,ActivemanagementandManipulationtrial(Farrin etal.2005)
Totesttheintegrityofthestudyprotocolusingaseriesofsubstudies
Plannedasclusterrandomisedtrial
3treatments activemanagement(practicelevel);spinalmanipulationandexercise(patientlevel)
Findings: Majorityofmethodsweresuccessful
Problemwithdifferentialrecruitmentbetween
practices changedtononclustereddesign
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
18/30
Pretrialmodelling
Example Fallspreventiontrial(Eldridgeetal.2005)
Toinformdesignandtestlikelihoodofthe
interventionbeingviableandeffective Costeffectivenessmodelusingpilotdata
o UsedprobabilitytreeandMarkovsimulation
Findings: Interventionwouldreduceproportionfalling
byonly2.8%over12months
Ifpolicymakerswerewillingtospend30,000perQALYgained,therewasstillonlya40%chancetheinterventionwouldbecost
effective
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
19/30
5.Selectionofappropriate
outcomemeasure(s)
Distinguishbetweenprimaryandsecondary
outcomemeasures Validandreliable(repeatable&reproducible)
Directlymeasuredvs patientreported
o Includeadditionalobjectivemeasureswhenselfreportingmaybeunreliableeg.selfassessedsmokingcessationandbiochemicalmeasure
o HRQL usegenericanddiseasespecificmeasure
Selectmostappropriateoutcomeforevaluatingtheeffectivenessoftheintervention
eg.levelofkneepain, kneefunction,abilitytowork,satisfactionwithtreatment
Individuallevelvs group(cluster)level
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
20/30
6.
Recruitment
Successfulrecruitmentrequiresacoordinatedapproachandgoodpilotwork
Importanttoengagepracticesearlyono IsresearchquestionimportantforPrimaryCare?
o Whatisitsprioritycomparedtootherissues?
o Howdoesitimpactonpatientdoctorrelationship?
o IsGPconfidenttoraiseresearchissuewithinasensitiveconsultation?
Timeconstraintsareamajorissue
Needtofindefficientwaystoidentifythesampleandgainconsent
Complexinterventionscanhavedifferentlevelsofrecruitment(practices&patients)
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
21/30
Principlesofgoodrecruitment Engagewithallstakeholders(GPs,practicestaff
andparticipants) Brandfortrial(eg.BEAM,PANDA,SCAMPS)
Welldevelopedmarketingstrategy,goodPReg.BellsPalsytrialusedlocalcelebrityinmedia
Wellwrittenpatientinformationdocuments
InvitationtotakepartcomingfromownGP
UsetrainedstaffotherthanGPstoidentifyandconsentparticipantseg.practicenurses
Providestafftrainingindiseasetopicandresearch
GetsupportfromlocalPCRNinfrastructure ResearchReadyaccreditationscheme
ePCRN (www.ePCRN.org)
Reimbursepracticesfortakingpart NB.Participantsareallowedtooptout
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
22/30
7.Methodofrandomisation
Byindividualorbyclustereg.GPpractices,households,nursinghomes
o relativecostsandjustification
Relativelyfewerclustersthanindividualsare
usually
available
higher
prob.
of
imbalanceo inthesizeofeachtreatmentarm
o inbaselinecovariatedistributionsatindividuallevel
Complexinterventionsinprimarycaremayhavemultiplecomponentseg.simpleparalleldesignvs factorialdesign
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
23/30
Imbalanceinsizeoftrt groups
Tooptimisepowerneedtoensure
o
an
equal
number
of
clusters
in
each
treatment
armo anequalnumberofpeopleineachtreatmentarm
Toensurebalanceinnumbersofpeopleineach
armcanuseblockingo interventionsareassignedrandomlywithineachblock
o varyingblocksizesreducespredictabilityofnext
assignment Allocationconcealmentisharderinclustertrials
o eachclustergetssameallocation
o useofplacebosisnotusuallyfeasible
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
24/30
Imbalanceinbaselinecovariates
Imbalancemayaffectface validityofcomparisonsandoverallconclusions
Waystominimiseimbalance: Adjustmentbyanalysis mayresultindifferent
unadjustedandadjustedestimatesoftreatmenteffectso byeffectsizeandsignificance
o difficultiesininterpretation
Atthedesignstage byidentifyingselected
covariateswhichmaybeimportantpredictorsofoutcomeo Randomiseusingstratification prepareaseparate
randomisationscheduleforeachstrata
o Useminimisation handleslargernumberofselectedvariables
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
25/30
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
26/30
Samplesize
Identifyprimaryoutcomemeasureand
calculatesamplesizeforindividualtrial
FindestimateofIntraclusterCorrelation
Coefficient(ICC)
o Fortrialsrandomisinggeneralpracticeswith
patientleveloutcomes,ICCsusuallyaround0.05.
o PapershavebeenpublishedprovidinglistsofICCs
Multiply(inflate)samplesizebydesigneffecto 1+(m1)xICC wheremisclustersizeassumingall
clustersizesareequal
Pre2000manyCRTswereunderpowered
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
27/30
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
28/30
Example DESMONDtrialComparisonof4methodsofanalysis:outcomeistheproportionof
patientswithanHbA1cbelow7%, interventionisstructurededucn
Model OddsRatio
StandardError
z P > | z | (95% Confidence Interval)
Cluster
specific
1.085539 0.166037 0.54 0.592 (0.804362, 1.465007)
Population averaged:
Robust 1.161681 0.271156 0.64 0.521 (0.735194, 1.835573)
Independent
errors
1.161681 0.162818 1.07 0.285 (0.882643, 1.528933)
Exchangable
errors
1.079769 0.160086 0.52 0.605 (0.807480, 1.443876)
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
29/30
10.Conclusion
Presentedmainstatisticalissuesforconductingcomplexinterventions
ProvidesaflavouroftheissuescoveredinourPHCSGmeetingsoverpast8years
Balancebetweenmethodologicalissuesandmorepracticalissuesofrealliferesearch
o manyissuesnotuniquetoprimarycaresetting
Challengeremainsofmaintainingandexpandingthecapacityofbothmethodologicalandappliedexpertiseinprimarycare
8/6/2019 Oxford Talk SMMR Paper July 2011 Final
30/30
Reference LancasterG.A.,CampbellM.C.,EldridgeS.E.,Farrin
A.,
Marchant M.,
Muller
S.,
Perera R.,
Peters
T.J.,
PrevostA.T.,Rait G.(2010).TrialsinPrimaryCare:statisticalissuesinthedesign,conductandevaluationofcomplex
interventions.StatisticalMethodsinMedicalResearch19:34977.
Facultyof1000publication.
Primstat data
archive www.jiscmail.ac.uk/primstat
presentationsandsummariesofdiscussionsfrommeetingsofPHCSG
http://www.jiscmail.ac.uk/primstathttp://www.jiscmail.ac.uk/primstat