EnsuringActiveImplementationSupportforCounties&CommunitiesScalingthe

TriplePSystemofInterventions

May2018

2

Acknowledgements&Disclosure

Developmentofthisdocumentwassupportedbythreefundingsources:

(1) TheDukeEndowmentGrantAgreementNo.1945-SP,UtilizingCountyEvaluationFindingstoBuild ImplementationCapacityand Infrastructure to Support theTripleP SystemofInterventionsinNorthCarolina.

(2) TheNorthCarolinaDepartmentofHealthandHumanServices,DivisionofPublicHealth

ContractNumber00034755,UtilizingCountyEvaluationFindingstoBuildImplementationCapacity and Infrastructure to Support the Triple P System of Interventions in NorthCarolina–DPH.

(3) TheNorthCarolinaDepartmentofHealthandHumanServices,DivisionofSocialServices

ContractNumber00034805UtilizingCountyEvaluationFindingstoBuildImplementationCapacity and Infrastructure to Support the Triple P System of Interventions in NorthCarolina–DSS.

RonPrinz,Ph.D.,isaconsultanttoTriplePInternational,whichisthetechnologytransferentity

commissionedbytheUniversityofQueenslandtodisseminatetheTriplePsystem,andtotheCenters

forDiseaseControlandPrevention,whichisinvolvedinimplementation/disseminationprojectsrelated

toTripleP.

Suggestedcitation:Aldridge,W.A.,II,Boothroyd,R.I.,Veazey,C.A.,Powell,B.J.,Murray,D.W.,&

Prinz,R.J.(2018,May).EnsuringActiveImplementationSupportforCounties&CommunitiesScalingtheTriplePSystemofInterventions.ChapelHill,NC:TheImpactCenteratFrankPorterGrahamChild

DevelopmentInstitute,UniversityofNorthCarolinaatChapelHill.

©2018WilliamA.AldridgeII,RenéeI.Boothroyd,ClaireA.Veazey,ByronJ.Powell,DesireeW.Murray

andRonaldJ.Prinz.

3

TableofContentsAcknowledgements&Disclosure...................................................................................................2

Background....................................................................................................................................4

ImplementationScience.............................................................................................................4

NorthCarolinaTriplePImplementationEvaluation(TPIE)........................................................5

Co-CreationPartnerships...........................................................................................................6

ImplementationSupportforCountiesorCommunitiesScalingTripleP.......................................6

ObjectivesofImplementationSupport......................................................................................6

PrinciplesofImplementationSupport.......................................................................................7

Population-levelOutcomes........................................................................................................9

TriplePSystemOptimization...................................................................................................10

ImplementationPerformance..................................................................................................13

LocalImplementationCapacity................................................................................................14

Co-CreationPartnerSupport....................................................................................................15

ProvidingImplementationSupporttoCountiesorCommunitiesScalingTripleP.......................18

AlignmentofSupportamongTriplePAmerica&IntermediaryOrganizations.......................20

AStage-BasedApproachtoSupportingtheScale-upofTriplePinCountiesorCommunities23

ACoreStoryofImplementationSupport,FlexiblyApplied.....................................................34

APPENDIXI:RecommendedToolstoSupportImplementation&Scale-UpProcesses...............36

APPENDIXII:RecommendedMeasuresofImplementation&Scale-Up......................................42

APPENDIXIII:RecommendedMeasuresandRecordsofImplementationSupportQuality........53

References....................................................................................................................................56

4

BackgroundEvidence-basedpreventionandwellbeingprogramsofferagreatdealofpromisetosupportthehealth

andwellbeingofchildren,youth,families,andcommunities.Infact,manyfundersandserviceproviders

are shifting toward models that have demonstrated positive impact through rigorous evaluations.

However,implementingandscaling-uptheseinnovationscanbeachallengeinthecontextofbusinessas

usual.Despitebestintentions,longstanding,complexservicesystemshaveatendencytopullinnovation

back to past practice. This challenge canprevent evidence-based strategies fromachieving expected

outcomes.

The Implementation Capacity for Triple P (ICTP) projects aim to develop methods, materials, and

opportunitiestosupportstatewidepartnersandlocalcommunitiestosuccessfullyandsustainablyscale

the Triple P – Positive Parenting Program (Triple P) system of interventions so that population-level

benefitsarerealizedforlocalchildren,families,andcommunities.Toaddresstypicalchallengesrelated

toimplementationandscale-up,ICTPprojectsleverage:

(1) Currentresearchandbestpracticesfromimplementationscience,

(2) Mixed-methodsevaluationfindingsfromtheNorthCarolinaTriplePImplementationEvaluation

(TPIE),and

(3) Partnershipswithstatewidestakeholdersinvolvedinscaling-uptheTriplePsystem.

ImplementationScienceCurrentresearchandbestpracticesfromimplementationscienceprovidethebackboneforICTPprojects.

Researchandappliedlearningfromeffortstosuccessfullyimplementandscaleevidence-basedprograms

havebeenamassingoverthepasttwodecades.2-4Amongreadyorganizationsandsystems,developing

andsustaininglocalcapacityaroundcoreimplementationprocesseshaveemergedasessentialpartsof

successandsustainability.4,5Althoughanumberofimplementationscienceframeworksarenowavailable

tomakesenseofkeyimplementationconcepts,themostpromisingapproachestoimplementationand

scale-upgivestrongattentiontothreekeyfeaturesoflocalimplementationcapacity:

(1) Linkinglocalleadershipandimplementationteamswithin(e.g.,individualserviceagencies)and

across(e.g.,communitycoalitions ledby localbackboneorganizations,statewide intermediary

organizations,andstateserviceagenciestosupportimplementation)levelsofcommunityservice

systems;2,4,6-20

(2) Bestpracticesforpractitioners’professionaldevelopment(i.e.,selection,training,coaching)to

deliverprogramsasintendedandwithexpectedbenefitsforchildrenandfamilies;21-34and

ThepurposeofthisimplementationsupportplanistoprovidedetailedinformationtostateandlocalTriplePcoordinators,funders,policymakers,andtechnicalassistanceprovidersaboutthecoreprinciples,partnerroles,coordinatedactivities,andoutcomesfortheprovisionofactiveimplementationsupporttocountiesandcommunitiesscalingtheTriplePsystemofinterventions.

5

(3) Quality and outcome monitoring for systems or organizational improvement and program

optimization.4,27,35-42

For communitywide prevention andwellbeing efforts, developingmedia and networking strategies to

mobilize knowledgeandbehavior changealso appears tobe important for achievingpopulation-level

outcomes.14,43-49ICTPprojectsmakesystematiceffortstoblendleading-edgeimplementationscienceand

bestpracticesintoallevaluationandimplementationsupportactivities.

NorthCarolinaTriplePImplementationEvaluation(TPIE)Tobringthescienceofimplementationclosertotheground-level,ICTPprojectswerealsofoundedon

twoimplementation-science-basedevaluationsofTriplePinNorthCarolina:TPIEandTPIE-Qualitative.

FromJanuary2014throughDecember2015,TPIEevaluatorsexaminedtheimplementationandscale-up

of theTripleP system inCabarrusandMecklenburgcounties.Specifically, thepurposeofTPIEwas to

evaluate capacity and infrastructure for the active implementation of Triple P to inform the planning

process for impact and sustainability. In late winter and early spring 2016, the TPIE team added a

qualitative evaluation component (TPIE-Qualitative) to better understand the findings from the initial

implementation evaluation and further improve the planning process for Triple P impact and

sustainability. Although highlights of evaluation results are touched upon in this section, detailed

evaluationbackgrounds,evaluationfindings,andlistsofevaluators’recommendationsareavailableinthe

TPIEFinalReport50andtheTPIE-QualitativeReport,51bothofwhicharelocatedontheICTPwebsiteat

https://ictp.fpg.unc.edu/lessons-learned.

TPIE results50 highlighted several strengths of local Triple P implementation capacity in Cabarrus and

Mecklenburgcountiesduringtheevaluationperiod,includingthecapacityofcountyTriplePleadership

teams, the capacity of Cabarrus County’s Implementation Team, counties’ Triple P practitioner

recruitmentandselectionprocesses,localTriplePtrainingprocesses,andcounty-levelTriplePdecision-

supportdatasystemsandqualityimprovementprocesses.Inadditiontothesestrengths,fourareasof

implementationcapacityneededparticularattentionandfurtherdevelopment:agencyimplementationteam structures; infrastructure to support Triple P practitioners’ ongoing coaching followingaccreditation; Triple P fidelity assessment practices; and infrastructure for using Triple P data andfeedbackaboutimplementationbarriersandfacilitatorsforagencyTriplePqualityimprovement.

Four risk factors for service agencies’ discontinuationof Triple P implementationwere also identified

duringTPIE,includinghaving:

(1) Only one Triple P practitioner within the service agency (this does not pertain toprivate/independentTriplePpractitioners);

(2) LessdevelopedagencyTriplePleadershipandimplementationteamstructures;

(3) AlesshospitableagencyimplementationclimateforTripleP,whichmaybeindicativeoflower

agencyleadershipandmanagementsupportforTripleP;52and

(4) LessformalizedordocumentedagencyTriplePsustainabilityplans.

At the county-level, TPIE results also suggested that the successful scale-up of Triple Pmay bemore

challenging if the county implementation team has less capacity (particularly in terms of formally

allocatedtimeandeffortforteammembers),thecountydoesn’thaveadequatefinancialresourcesto

6

supportlocalTriplePscale-up,and ifthecountypreventionsystemandpopulationarelargeroraremorecomplex.

TPIE-Qualitativeresults51reinforcedmanyoftheseinitialTPIEfindingsandaddedahandfulofadditional

important points about Triple P scale-up in NC counties. These included: the need for more activeimplementationsupporttocountiesscaling,andagenciesimplementing,TripleP;theneedforrobustexplorationandreadinessprocesses ateach levelof thestatewideTripleP systembeforeembarking

uponlocalTriplePadoptionorinstallingnewfeaturesofTriplePimplementation;thebenefitsofusingacoalition approach to locally scaling Triple P in NC counties and ensuring a statewide learningcollaborativeforcountyTriplePcoordinators;andtheneedformoreactivelyandpurposefullyinvolvingcommunitymembersintheTriplePimplementationinfrastructure.

Co-CreationPartnershipsFinally, ICTPprojectsweredevelopedaroundaco-creationmodelofapplying implementationscience

withinlocalcontexts.Whilethescienceofimplementationprovidesmeaningfuldirection,theutilization

ofstrategiesfromimplementationscience,andthedevelopmentoflocalimplementationinfrastructure,

requiresco-creationfromfivepartners:53

(1) Serviceagencyleadershipandstafffromimplementingsites;

(2) State/localfundersandpolicymakers;

(3) Intermediaryandpurveyororganizationsthatprovideimplementationandprogramsupport(i.e.,

implementationtechnicalassistanceproviders,TriplePAmerica);

(4) Activeandinvolvedcommunitymembers(e.g.,communityparentsandyouthbeingserved);and

(5) Interventiondevelopersandpreventionscientists.

The successful and sustainable scale-up of Triple P in communities statewide and the realization of

population-levelpreventionandwellbeingbenefitswillnecessitatecollaborativepartnershipsamongall

fiveco-creationpartners.ICTPprojectsrespondtoopportunitiesforco-creationandhumblyacceptthat

theworkofimplementingandscalingTriplePcannotbeaccomplishedbyoneortwoofthesepartners

alone.

ImplementationSupportforCountiesorCommunitiesScalingTriplePThepurposeofthisimplementationsupportplanistodetailcoreprinciples,processes,features,partner

roles, and intended outcomes for the provision of active implementation support to counties or

communitiesscalingtheTriplePsystemofinterventions.Indoingso,theplantakesacustomizableand

adaptiveapproach54,55tosupportingimplementationprocessesratherthanprescribingaseriesofspecific

stepsandprocedures.

ObjectivesofImplementationSupportActiveimplementationsupportprovidedtocountiesscalingtheTriplePsystemofinterventionsseeksto

contributetoseveralobjectives.

(1) Strengthening amulti-level system of implementation and program support from state to

countiestoagenciestopractitionerstofamilies.

7

(2) Organizing and aligning communitywide implementation capacity. This includes ensuringadequate implementation capacity within lead or backbone agencies and service agenciesparticipatinginlocalTriplePcoalitions.

(3) Supporting implementation performance across lead/backbone agencies and local Triple Pserviceagencies.

(4) LocallyscalingtheTriplePsystemtorespondtoidentifiedcommunityneeds,characteristics,and

readiness.

(5) Supporting practitioners’ delivery of Triple P interventions as intended and in response toparents’needsandpreferences.

(6) Increasingtheprobabilitythatintendedpreventionandwellbeingoutcomeswillbeachievedatscale.

(7) SustainingTriplePimplementationandprogramperformance.

PrinciplesofImplementationSupportAcrossallpartners,theprovisionofimplementationsupportforcountiesscalingtheTriplePsystemof

interventionsbenefitsfrombeingguidedbyseveralprinciples.1

(1) Proactivesupport:Likeothereffortstochangeindividualandgroupbehavior,implementingand

scalingevidence-basedpreventionstrategiesrequiresintentionalandfocusedsupport.Proactiveimplementationsupportanticipatesneedsandincorporatesstrategicapproachestobringnew

knowledge, skills, and opportunities for recipients to apply and test new learning – with

reinforcementandsupportivefeedback–intheirownsystemsenvironments.Suchlearningand

supportisoftennecessaryatindividual,team,organizational,andsystemlevels.

(2) Implementationscienceandbestpractices:Asweaskcommunityleadersandlocalpractitioners

tobeguidedbythescienceofprevention,sotoomustco-creationpartnersinvolvedinsupporting

implementationandscale-upbeguidedbythescienceofinnovationimplementation.Arangeof

frameworksandtoolsarenowavailablefromimplementationsciencetosupporttheintroduction

ofkeyconceptsandstrategiesforeffectiveimplementation.Furthermore,theseframeworksand

tools can inform exchanges of ideas with local stakeholders to enable local application and

sustainability.

(3) Co-creation:Thedevelopmentoflocalimplementationinfrastructureisbecomingrecognizedas

a process of co-creation.53,56 Within the co-creation framework, five partners contribute to

successfulandsustainableimplementationandscale-up:

a. Serviceagencyleadershipandstafffromimplementingsites;

b. State/localfundersandpolicymakers;

c. Intermediary and purveyor organizations that provide implementation and program

support;

d. Active and involved community members (e.g., community parents and youth being

served);and

e. Interventiondevelopersandpreventionscientists.

Support for active implementation and scale-up becomes stronger as collaborations and

contributionsamongthesefivepartnersincrease.

8

(4) Contextualized and responsive support: While the science of implementation provides

meaningful grounding for any implementation effort, to increase chances for success and

sustainability,effortstoimplementandscale-uppreventionprogramsmustbeoptimizedwithin

localcontexts.17,57-70Implementationstrategiesneedtobeconsideredandtailoredaccordingto

key features of local prevention systems, such as size, history, resources, culture, population

density, and political and social complexities. Furthermore, ongoing implementation support

needstoberesponsivetolocalprogress,setbacks,feedback,andkeyevents.

(5) Adaptive leadership: Implementation and scale-up are adaptive processes, not technical

processes.71 Implementation support partnersmust develop an appreciation for, and comfort

with, thediverseperspectivesheldwithin local systemsenvironments andbegin to recognize

theseascluestothepresenceofadaptivechallengesembeddedwithinthesystemanditspeople.

HeifetzandLaurie72putforwardsixprinciplesofadaptiveleadershipthatcanbeusedtomanage

adaptivechallenges:

a. Get on the balcony: step back from daily system operations to see larger patterns of

individual and collective behavior and local history that may be either facilitating or

hinderingthesystems’willingnessorabilitytochange.

b. Identify the adaptive challenge: take time to clearly define adaptive challenges.

Definitionsshouldtakeintoaccountanunderstandingoflocalpeople,organizationaland

communityhistory,largersystempressures,andidentifiedsourcesofconflict.

c. Regulate distress: create a functional balance of system stress by using conflict as an

opportunity for learning and creativity, sequencing and pacing work, and preventing

stakeholdersfromfeelingoverwhelmedbychange.

d. Maintain disciplined attention: maintain focus on tough questions and prevent the

avoidanceofadaptiveworkrecognizedbyslidingbackintofamiliarroutinesorengaging

peripheralissuesortopics.

e. Givetheworkbacktopeople:buildthecollectiveproblem-solvingconfidenceofsystem

stakeholdersratherthanprovideexpertsolutionsorlettheburdenofadaptiveworkfall

onthefewidentifiedvocalleaders.

f. Protectvoicesofleadershipfrombelow:ensurethattheexperiencesandideasofthoseoften marginalized in change initiatives, including front line staff and community

members,arevoicedandplayanequalroleingeneratingsolutionssothattheywillbe

themostsuccessfulandsustainable.

(6) Stage-basedapproach:Because implementationandscale-upareadaptiveprocessesoftenwithin

complex systemenvironments, they require iterative series of inquiries, actions, and adjustments

often across longer-term engagements.27 Stage-based approaches to implementation have been

widely utilized to address these demands,12,27,73-75 and implementation support partnersmust be

mindfultopaceandmodifysupportactivitiesacrosssuchstages.

a. Readiness&Exploration:ensuringthatsystempartnersarewillingandabletoproceedwith

implementation efforts designed tomeet their goals and best fit to their local strengths,

needs,andpreferences.

b. Capacity Development: ensuring that local systems and partners have the resources and

abilitiesshownbyimplementationsciencetosupportsuccessfulandsustainablescale-up.

9

c. SupportedPerformance:providingsupportivebehavioralcoachingtosystempartnersasthey

begintomanageimplementationprocessesanddevelopconfidenceinbroadlyapplyingnew

skillsandabilitiesintheirlocalenvironments.

d. Local Regulation: supporting system partners to manage implementation and scale-up

autonomously,withreducedexternalsupport,sothateffortsaresustainedprimarilybylocal

capacitiesandperformance.

Stagesof implementationarenotmeanttoconvey linearity.Rather, implementation iswidely

believed to be a nonlinear process requiring cycling back and forth between stages of

implementationasnewelementsofworkariseandadaptivechallengesareidentified.27,73

(7) Data-drivenprogressmonitoringandimprovement:Asadvocatesforthetranslationofevidenceintopractice,implementationsupportpartnerscollectandusedatatoidentifylocalneedsand

planresponsivesupportstrategies,monitortheprogressandoutcomesoflocalimplementation

efforts,monitortheeffectivenessoftheirownsupport,andmakequalityimprovementsbased

ondataovertime.

(8) Localownershipofprogress: Implementationsupportpartnersshouldpromote local systems’

ownershipofimplementationprocessesandsuccesses.Althoughexternalpartnerscanbeseen

as instrumental to increasing implementation resources and abilities, ongoing success in

implementationandscale-upshouldnotbeperceivedtobedueto,ordependenton,external

supportpartners.Thisprinciplecanbedemonstratedbycontinuallypromotingcollective-efficacywithincommunitypreventionsystems.

ICTPIntegratedTheoryofChangeDrawingheavilyonChinmanandcolleagues’implementationTAlogicmodel,76atheoryofchangethat

describestherelationshipsbetweenkeyintermediateandlong-termoutcomesofactiveimplementation

supportisprovidedinFigure1.TheICTPIntegratedTheoryofChangeforsupportingtheimplementationand scale-upof theTripleP systemof interventions toachievepopulation-leveloutcomes is rathercomprehensiveandnotmeanttosuggestaprescribedprocessforscalingTripleP.Rather,withincertainlimits,localcommunitiesandsystempartnersmightcustomizetheirlevelofuseofthistheoryofchange.

Forexample,localcommunitiesmaychoosetomonitoronlycertainpopulation-levelorimplementation

outcomesarticulatedinthistheoryofchange.Likewise,co-creationpartnerrolesmayvaryinintensity

andfunctionaccordingtolocalcommunitycontext.Itisrecommended,however,thatallcommunitiesestablishcollaborativerelationshipswiththenotedco-

creationpartners,asfeasible.Also,attendingtoeachcomponentoflocalimplementationcapacityand

performancearticulatedinthetheoryofchangemaybeessentialtorealizingthefullimpactofTripleP

onpopulation-leveloutcomes.

Population-levelOutcomesAsapartofstatewideevaluationactivities,systempartnersoftenagreetomonitorsomeorallofthe

population-leveloutcomevariablesthatdemonstratedresponsivityinpriorresearchonthecountywide

scale-upoftheTriplePsystemintheSoutheasternUnitedStates:77

(1) substantiatedchildabuseandneglect,

(2) out-of-homefostercareplacements,and

10

(3) childinjuriestreatedinahospital.

Inadditiontoanyrequiredstatewideevaluationoutcomevariables, localTriplePcoalitionsmayhave

interestandresourcestomonitorotherchild,family,andcommunityoutcomesthathavedemonstrated

responsivitytoTriplePuse.TripleP,boththroughindividualinterventionsandtheaggregatesystem,has

demonstrated positive child and family outcomes across a number of research and evaluation trials

globally.LocalTriplePcoalitionsmaybenefitfromexaminingthefullTriplePevidence-base,availableat

https://www.pfsc.uq.edu.au/research/evidence/.ReaderscanquerytheTriplePevidence-baseaccording

toindividualTriplePinterventionsaswellaskeytopicsandoutcomevariables.

ICTPprojectspromotetheideathatinterventionoutcomes,whetherindividualorpopulation-level,can

beoptimizedinalocalcontext.35WehopethatstakeholdersinvolvedincountyTripleProlloutswilltake

advantageofthisperspectiveandstrivetomovebeyondthelevelofoutcomesestablishedinpriorTriple

Presearchforthebenefitoflocalcommunities.

TriplePSystemOptimizationReviewsoftheresearchliteraturehavemadeclearthatimplementationqualityimpactstherealizationof

outcomeswhenevidence-basedpreventionandwellbeingprogramsareusedintherealworld.78Perhaps

themost recognized feature of implementation quality is fidelity to the intended delivery of adoptedprograms. However, several other implementation outcomesmay also be important, particularly as

relatedtoachievingfavorableserviceandclientoutcomesatscale.Forexample,Proctorandcolleagues79

offer eight core implementation outcomes: acceptability, adoption, appropriateness, cost, feasibility,

fidelity, penetration, and sustainability. The ICTP Integrated Theory of Change for supporting the

implementationandscale-upoftheTriplePsystemof interventionsadaptsandincorporatesessential

featuresoftheseeightimplementationoutcomes,andincludesotherimplementationoutcomesthatmay

beofparticularinterestgivenTripleP’smodel,history,andongoingaimsincounties:

(1) Accessibility. Progressing beyond Proctor and colleagues’ simpler adoption outcome,

accessibility is defined as the degree towhich local families can access parenting and family

supportinaccordancewiththelevelofsupporttheyneedorprefer.

(2) SystemAlignment.NotrepresentedinProctorandcolleagues’originallistbutimportantforany

system of interventions, system alignment is defined as the degree to which local serviceagenciesor individual interventionswork inconcert towardscollectivewellbeinggoals rather

thaninsilosorfragmentation.

(3) Feasibility.AsdefinedbyProctorandcolleagues(p.69),feasibilityistheextenttowhichanewtreatment,oran innovation,canbesuccessfullyusedorcarriedoutwithinagivenagencyor

setting.Feasibilityhasalottodowithwhetherornotthelocalsettingofcarehasthenecessary

financial, human, and implementation resources to support delivery of the intervention as

intended.

(4) Appropriateness.AsdefinedbyProctorandcolleagues(p.69),appropriatenessistheperceivedfit,relevance,orcompatibilityoftheinnovationorevidence-basedpracticeforagivenpractice

setting,practitioner,orconsumer;and/orperceivedfitoftheinnovationtoaddressaparticular

issueorproblem.

11

Figure1.ICTPIntegratedTheoryofChangeforsupportingtheimplementationandscale-upoftheTriplePsystemofinterventionstoachievepopulation-leveloutcomes

12

(5) Fidelity. As defined by Proctor and colleagues (p. 69), fidelity is the degree to which aninterventionwasimplementedasitwasprescribedintheoriginalprotocolorasitwasintended

by the program developers. Four dimensions of program fidelity relevant to Triple P in

communities are detailed by Dane and Schneider80 and later reinforced byMihalic,

81 (p. 83)

relativetopreventionprograms:

a. Adherence refers towhether the intervention isbeingdeliveredas itwasdesignedorwritten (i.e.,with all core components beingdelivered to the appropriatepopulation;

stafftrainedappropriately;usingtherightprotocols,techniques,andmaterials;andin

thelocationsorcontextsprescribed).

b. Qualityofprogramdeliveryisthemannerinwhichapractitionerdeliversaprogram(e.g.,

skill in using the techniques or methods prescribed by the program, enthusiasm,

preparedness,andattitude).

c. Caregiverengagementistheextenttowhichparticipantsareengagedbyandinvolvedintheactivitiesandcontentoftheprogram.

d. Dosagemayincludeanyofthefollowing:thenumberofsessionsimplemented,lengthof

eachsession,orthefrequencywithwhichprogramtechniqueswereimplemented.

Dane and Schneider and, separately, Mihalic also discuss the fidelity dimension programdifferentiation,whichmaybemoreimportantwithincontrolledresearchsettingsthanTripleP

scale-upincommunitiesstatewide.

(6) Acceptability:AsdefinedbyProctorandcolleagues(p.67),acceptabilityistheperceptionamong

implementation stakeholders (e.g., families) that a given practice or program is agreeable,

palatable,orsatisfactory,asdelivered.

(7) Reach: Proctor and colleagues use a synonymous term,penetration,which is defined as theintegration of a practicewithin a service setting and its subsystems (p. 70).Reachmight be

measuredby(a)thenumberofpeoplewhoreceiveaninterventioncomparedtothosewhoare

eligibletoreceivetheintervention,or(b)thenumberofpractitioners(actively)deliveringthe

interventioncomparedtothenumbertrainedinorexpectedtodelivertheintervention.TPIE

resultsandexperiencefromTriplePstakeholdersinNCsuggestthattherehasbeenasignificant

discrepancy between the number of practitioners trained in Triple P and thosewho remain

activelydeliveringTriplePinterventionstolocalfamilies.

(8) Cost: As defined by Proctor and colleagues (p. 67), cost is related to the cost impact of an

implementationeffort.Proctorandcolleaguesnotethreecostcomponentsmaybeofinterest:

a. costsofdeliveringtheintervention,

b. costsoftheimplementationstrategiesthatwillbeusedtosupporttheintervention,and

c. costvariabilityassociatedwiththelocalservicedeliverysetting.

Anadditionalvariablerelatedtocost,returnoninvestment,hasreceivedincreasinginterestandattentionrelativetotheimplementationandscale-upofevidence-basedpractices

82.

13

(9) Sustainability:AsdefinedbyProctorandcolleagues(p.70),sustainabilityistheextenttowhichanewly implemented intervention ismaintainedor institutionalizedwithinaservicesetting’s

ongoing,stableoperations.

SystemstakeholdersinvolvedindifferentlevelsofcommunityTripleProllouts(e.g.,state,county,agency,

and practitioner) may have varied interest across these nine implementation outcomes. While

stakeholdersmaywanttoreviewthesealternativesanddeterminewhichmixmaybeofmostinterest

andusabilityattheirsystemlevel,theICTPprojectsteamstronglyrecommendsthatprogramfidelityismonitored by every system level. Fidelity has demonstrated particular importance in relation to the

replication of evidence-based program outcomes in real world implementation.78,81

In addition, by

choosing from and attending to other implementation outcomes, such as acceptability andappropriateness, we believe that system stakeholders at any level can monitor implementation in

accordance with Triple P’s stated philosophy of “fidelity and flexibility.” Monitoring variables like

acceptability and appropriateness can ensure that interventions core components are reaching local

familiesinawaythatisresponsivetofamilyneedsandpreferences.

Finally,ICTPprojectspromotetheideathatimplementationoutcomes,likepopulation-leveloutcomes,

canbeoptimizedinlocalcontext.35Hence,werefertothissectionoftheICTPIntegratedTheoryofChange

asTriplePSystemOptimizationtoreflecttheperspectiveofcontinuousqualityimprovementwithinlocal

contexts.

ImplementationPerformanceImplementation outcomes are influenced by the level of local implementation capacity and

performance.76Chinmanandcolleaguesdefineimplementationperformanceas“thelevelofqualityat

which [key implementation support practices] are carried out” (p. 3). The ICTP Integrated Theory of

Changedetailsfivecoreimplementationsupportpractices:

(1) Leading and supporting Triple P implementation and scale-up, including identifying and

addressingimplementationbarriersandspreadingsuccesses;

(2) DevelopingcompetentandconfidentTriplePpractitionerswhocandeliverTriplePwithfidelity

andflexibility;

(3) Gathering, analyzing, and reporting to the right people at the right times program and

implementationdatarelatedtoTriplePdelivery;

(4) System-widelearningandcontinuousqualityimprovementofTriplePimplementation,delivery,

andoutcomes;and

(5) Mobilizingknowledgeandbehaviorchangeacrosscommunitiesbeyondthatcreatedbydirect

serviceinterventions.

We believe these performance indicators provide congruity with core components of local

implementationcapacityasdescribed inthenextsection.Theseperformance indicatorsarehigh level

andmaybefurtherbrokendownintomorespecificperformancebehaviors.Forexample, leadingand

supporting Triple P implementation may involve executive leaders’ ongoing demonstration of

commitment to Triple P implementation (i.e., “implementation climate”) and aligning community

prevention strategies under common approaches and outcomes of implementation.8,83

Likewise,

14

developingcompetentandconfidencepractitionersmayinvolvehighqualitypractitionerrecruitmentand

selection,training,andcoachingpractices.8,27,83

LocalImplementationCapacityAs discussed earlier, ICTP projects take a perspective that the most promising approaches to

implementationandscale-upgivestrongattentiontothreekeyfeaturesoflocalimplementationcapacity:

(1) Linkinglocalleadershipandimplementationteamswithin(e.g.,individualserviceagencies)and

across(e.g.,communitycoalitions ledby localbackboneorganizations,statewide intermediary

organizations,andstateserviceagenciestosupportimplementation)levelsofcommunityservice

systems;2,4,6-20

(2) Bestpracticesforpractitioners’professionaldevelopment(i.e.,selection,training,coaching)to

deliverprogramsasintendedandwithexpectedbenefitsforchildrenandfamilies;21-34

and

(3) Qualityandoutcomemeasurementandmonitoringforsystemsororganizationalimprovement

andprogramoptimization.4,27,35-42

Figure2.IdealSupportModelfortheTriplePSysteminNorthCarolina.

For communitywide prevention andwellbeing efforts, developingmedia and networking strategies to

mobilize knowledgeandbehavior changealso appears tobe important for achievingpopulation-level

outcomes.14,43-49

ICTPprojectshelptoidentifyand/orrecognizeanyexistingmulti-levelsystemsofsupportforthescale-

up of Triple P that have already started to develop within a state. Such cascading models of

implementation supportmayprovideeffectiveways topromotemeaningful capacityand roleswithin

15

eachlevelofastatewidesystemandsupportoverallsuccess.12,27

Figure2presentsonestate’smulti-level

systemofTriplePsupport,withinwhichkeyfeaturesofimplementationcapacityandperformanceareto

be embedded. While feedback loops are presented between single levels of the system, it is

acknowledgedthatfeedbackfromandtoeachlevelofthesystemislikelyhappeningandisimportantfor

qualityimprovementandincreasingthelikelihoodofsuccess.

Co-CreationPartnerSupportCo-creation partners working collaboratively to support county Triple P rollouts contribute to the

developmentoflocalimplementationcapacityandperformance.53,56

Keyfeaturesofco-creationpartner

rolesfollowbelowyetarenotintendedtobecomprehensive.Individualco-creationpartnersmayservea

varietyofuniqueandsharedfunctionsandthenatureandintensityofpartnerrolesmayshiftandchange

overtimeasimplementationandscale-upprogresses.

Localagencyleadership&staffLocalleadershipandstaffwithinTriplePcoalitionleadagenciesandserviceagencieshaveakeyrolein

generating initial readiness for implementation and scale-up of Triple P. Local readiness forimplementationcanbeunderstoodasacombinationofstakeholders’commitmenttothechangeprocess

andtheircollectivebelief that theycanmakethechanges thatwillberequired.84Several factorsmay

influencelocalreadinessforchange,suchasthevalueplacedonmakingthechange,taskdemandsthat

maybeinvolved,resourceavailability,andrelevantsituationalfactors.

Beyondgenerating initial readiness forTripleP implementationand scaling, local leadershipand staff

mustcontinuouslyensureseveralotherleadershipandcoordinationfunctionsforactivelyimplementing

andscalingTripleP.8 Thosewithexecutive leadershipof implementingandscalingTripleP–whether

withincommunityTriplePcoalitionleadagenciesorindividualserviceagencies–maysupportsuccess

and sustainability by demonstrating ongoing commitment to the change process and changepartnerships, and by creating and nurturing appropriate opportunities for change within localorganizationsandsystems.

8

ThoseleadingthedevelopmentofthecommunityTriplePcoalitionmustalsoensurethatTriplePandrelated family service initiatives arewell aligned and usable by practitioners, coalition policies andagency practices facilitate delivery of Triple P interventions as intended, and system changes andsuccessesarewellcommunicatedacrossstakeholdersandcommunitymembers.

8

Finally, thosewith day-to-daymanagement responsibilities for Triple P program implementation and

scaling – whether within community Triple P coalition lead agencies or individual service agencies –

supportsuccessandsustainabilitybyensuringongoingbuy-inandreadinessforstage-basedscale-upofTriplePinthecommunity;organizing,aligning,andsustainingthenecessaryinfrastructuretosupportTripleP implementationwithin thecommunity;andbyactivelyusingdataandother information forqualityimprovementofTriplePimplementation.

8

These leadership and coordination functions can be institutionalized within leadership and

implementationteamstructuresattheagencyandcommunityTriplePcoalitionlevels.Furthermore,they

maybeessentialtothedevelopmentandsustainabilityoflocalimplementationcapacityandperformance

more broadly,8,12,27

and may support hospitable agency and coalition climates for implementing and

scalingTripleP.52

16

State/localfunders&policymakersState and local funders and policymakers have an important role in creating a nurturing systems

environmentforcountyTripleProllouts.Inparticular,keyfunctionsinclude:85

(1) Ensuringtheavailabilityofadequatefinancialresourcestodevelopnecessaryimplementation

capacityandsupportthedeliveryofTripleP,

(2) Ensuring adequate time and space to reasonably expect implementation and scale-up to

translateintopopulation-leveloutcomes,and

(3) Settingexpectationsand resources forqualityandoutcomemonitoringofTriplePacrossalllevelsoftheTriplePsystem.

Whenfundingcomesfromstateagencies,theymayalsoplayasupportiveroleincoordinatingstate-level

learning collaboratives, statewide implementation support teams, and statewide intervention

components(e.g.,media-basedinterventioncomponents).

TriplePAmericaTriplePAmericaistheU.S.-basedpurveyorofTriplePtraining,materials,andimplementationsupport.

Recently,TriplePInternationalpublishedtheTriplePImplementationFramework(TPIF),whichdetails

theirroleinsupportingTriplePimplementationandsustainability.86TPIFdetailsfivephasesofactivities

betweenTriplePAmericaandlocalservicesystemsadoptingTriplePinterventions.

(1) Engagement:InitialinteractionswithcommunitystakeholderstoexploreifTriplePisagoodfit

fortheirgoalsandcommunityneeds.

(2) Commitment and Contracting: Confirmation of the scope of Triple P implementation and

facilitationofwrittenagreementsfortraining,resources,andsupport.

(3) ImplementationPlanning:Collaborationoncreationofanimplementationplan,includingplans

for communications, training and accreditation, service delivery, quality assurance, and

evaluation.

(4) Training and Accreditation: Delivery of standardized training and accreditation process forpractitioners.

(5) ImplementationandMaintenance:Engagementinfeedbackcycleswithcommunitystakeholders

around service delivery, quality improvement, ongoing development, and sustainability

mechanisms.

Across these fivephases,TriplePAmericahelps tosupportpractitionerprofessionaldevelopment todeliverTriplePinterventionsasintended,assurequality,enableoutcomemonitoring,andcontributeto the development of local implementation capacity needed to support and improve local Triple P

implementation.

IntermediaryorganizationsIntermediaryorganizationsdiffer fromprogrampurveyors in that they support thedisseminationand

implementation of more than one evidence-based program or practice and, as such, have a more

expandedrolethanprogrampurveyors.87AsdefinedbyMettrickandcolleagues

88(p.3),anintermediary

organization:

17

“Supports service array development through implementation technical assistance, creative

financingoptions,training,coaching,education,continuousqualityimprovementmonitoring,and

outcomesevaluation.

[An intermediary organization] connects providers, state agencies, local jurisdictions, and

purveyors toensure thateffective implementation leads to improvedoutcomesandbuildson

existingsystemsreformefforts.”

Becauseintermediaryorganizationsareoftenmoreregionallylocatedtoimplementationsitescompared

to thenationalor internationalpresenceofprogrampurveyors, theyareable toserveseveralunique

functions.ThroughtheirrecentCenterofExcellenceLearningCommunityfundedbytheAnnieE.Casey

Foundation,Mettrickandcolleagues88detailfivecorefunctionsforintermediaryorganizations:

(1) Implementationsupportforevidence-basedprograms;

(2) Research,evaluation,anddatalinkingcapacity;

(3) Partnershipengagementandcollaboration;

(4) Workforcedevelopmentactivities(includingtrainingandcoaching);and

(5) Policyandfinanceexpertise.

Intermediaryorganizationsdonotreplicatetheroleofstateagenciesorprogrampurveyors,rathertheywork in concert with state agencies, funders, and program purveyors to support the achievement ofcommongoals.Wherefunctionsoractivitiesoverlapamonganyco-creationpartners,developingclear

agreements about roles and how to support synergistic, rather than contradictory work patterns,

becomesessential.

Dependingonstatewideneedsandgoals,ICTPprojectteamsmayworktoidentifyandbuildthecapacity

ofan intermediaryorganizationorapartnershipof commonorganizations to supportTripleP system

implementationinacrossastate.

LocalcommunitymembersLocalcommunitymembers,includingthechildren,youth,andfamiliesreceivingservices,playessential

roles inthesuccessfulandsustainable implementationofevidence-basedinterventions,particularlyat

scale. Respondents in the TPIE-Qualitative evaluation identified that local communitymemberswere

particularlyhelpfulby:51

(1) Providing feedback and supporting continuous quality improvement of Triple P delivery atagency,county,andstatelevels;

(2) CatalyzingTriplePengagementwithintheircommunitiesbyword-of-mouthadvertising,sharing

positiveexperiences,andtransferringlearningandparentingskillstoothercommunityparents

andstakeholders;

(3) ChampioningTriplePwithlocal,countyandstatewidestakeholders;andby

(4) Fullyparticipating inTripleP implementationstructures, suchasdecision-makingbodiesthat

selectwhichTriplePprogramstoadoptlocally.

18

Inaddition,Boothroydandcolleagues89detailfivefunctionsthatactive,involvedpartnershipsbetween

localservicesystemsandcommunitymemberscansupportduringimplementationandscale-up:

(1) Listening to learn about and begin to address historical trauma (historical maltreatment of

familiesinkeycommunitiesidentifiedbyculturalfactorssuchasraceorincomelevel),mistrust

ofagenciesandsystems,andotherlong-standingandinstitutionalbarrierstosafety,health,and

wellbeing;

(2) Working with community members to identify system barriers to improved outcomes for

childrenandfamiliesandimplementactionplanstoaddressthosebarriers;

(3) Collaboratingwithcommunitymemberstoestablishculturallyrelevantsupportsandservicestomeettheunderlyingneedsofchildrenandfamilies;

(4) Meaningfullyinvolvingcommunitymembersinpractitionerprofessionaldevelopmentactivitiesandcommunitydesignteamsforeffective,sustainedimplementation;and

(5) Ensuringpartnershipmeetings,forums,andfeedbackloopsaresustainedsothatcommunity

membersarecontinuouslyconnectedtoandhelpguideongoingpracticeandsystemchanges.

ParticipantsinTPIE-Qualitativesuggestedthat,overall,thereisaneedformoreactivelyandpurposefully

involvingcommunitymembersinlocalTriplePimplementationactivitiesanddecision-making.51

TriplePdevelopers&researchersFinally, Triple P developers and researchers have both proactive and reactive roles relative to the

implementationandscale-upofTripleP.Proactively,TriplePdevelopersneedtoensure thatTriplePprogramsandstrategiesareusablewithincommunitypreventionsystems.12,21,90,91 Interventionsthatmeet usability criteria are regarded as teachable, learnable, doable, repeatable, and assessable in

practice.90,91

Triple P researchers also have a key role to ensure that Triple P programs and mediastrategiesareandremainevidence-based.ThiswasoneofthemostwidelyidentifiedrolesofTripleP

researchersduringtheTPIE-Qualitativeevaluation.51AsidentifiedinTPIE-Qualitative,TriplePresearchers

also have ongoing roles aroundmaking the Triple P evidence-base accessible and usable to localimplementationstakeholdersandforusingnaturallyoccurringimplementationeffortsasopportunitiestotesteffectiveimplementationstrategiesrelatedtoTripleP.

ProvidingImplementationSupporttoCountiesorCommunitiesScalingTriplePWhile all co-creation partners have essential roles in the implementation and scale-up of Triple P,

providingactive implementationsupportdirectlytosystemstakeholders isacorefunctionparticularly

related to Triple P America and intermediary organizations. These direct implementation supportprovidersareaprimarymechanismforcontributingtothedevelopmentoflocalimplementationcapacity

andperformance.

19

TheICTPIntegratedTheoryofChange(seeashort-handedversioninFigure3)remindsusthatcapacity

withoutperformancedeniesthepossibilityofimplementationoptimizationandtherealizationofchild

and family outcomes. Likewise, performance without adequate capacity may result in temporary,

inefficient, and suboptimal outcomes. One helpful way to think about this is that both the pipeline

(implementation capacity) and the water flowing through the pipeline as intended (implementation

performance) are essential. As such, although there are seven objectives of implementation support

articulatedinthefirstsectionofthisplan,theprimarygoalsofdirectimplementationsupportproviders– such as Triple P America and intermediary organizations – are meaningful contributions to thedevelopment of strong local implementation capacity and performance. Strong local capacity andperformanceofferthefoundationonwhichtherealizationofotherimplementationsupportobjectivesandlocalTriplePsystemoptimizationgoalscanbemosteffectivelyandsustainablyachieved.

TriplePAmericaandintermediaryorganizationsjoinmulti-levelsystemsofsupportforthescale-upof

TriplePinstatewidecontexts(seeFigure4).Stateleadership,localleadership,andimplementationteams

withincommunityTriplePcoalitionsworkasinternalchangeagentsinthismulti-levelsystemofsupport;

theyworkfromwithinstateandcountyservicesystemenvironmentstodevelopTriplePimplementation

capacityandperformance.Incontrast,TriplePAmericaand intermediaryorganizationsactasexternalchangeagents; theywork fromoutside stateandcounty service systemenvironments to support the

developmentofimplementationcapacityandperformance.1,65,92

Carefullydesigned,proactive,andongoingimplementationsupportfromexternalchangeagentshasbeen

identifiedasakeycomponentforachievingsystem-levelimpact,andisconsideredmosteffectivewhen

it contextualizes implementation strategies for local systems and works at multiple levels.17,57-70

The

followingsectionsofferdetailsaboutthealignmentofsupportfromTriplePAmericaandintermediary

organizationsandaflexible,stage-basedapproachtotheuseofcorepracticecomponentsforexternal

implementationsupporttostrengthenTriplePimplementationcapacityandperformanceincommunities

statewide.

Figure3.Directimplementationsupportroleandgoals.

20

Figure 4. TPA and IntermediaryOrganization integration into the ideal supportmodel for the Triple Psysteminastatewidesetting.

AlignmentofSupportamongTriplePAmerica&IntermediaryOrganizationsFigure 5 presents an integrated model of implementation support, aligning more generic localimplementationprocesses (i.e., notprogram specific)with thephasesof theTripleP Implementation

Framework(TPIF).86IntermediaryorganizationsandTriplePAmericamayworkinconcerttosupportthe

blendofgenericimplementationprocessesandTripleP-specificimplementationprocesses.Forexample,

during the exploration stage,12,27

lead agencies in communities and their partners will benefit from

assessingcommunitywellbeingneedsandcommunitysystemmembers’readinesstoimplementpractice

or program changes. These generic implementation processes may be supported by intermediary

organizationsasneeded.Onceneedsandreadinessforimplementationareclarified,contactmaybegin

betweentheleadcommunityagencyandTriplePAmerica(TPA)toassessthefitofTriplePwithidentifiedcommunityneedsandlocalreadiness(i.e.,theengagementphaseofTPIF).Ifadecisionismadetomove

forward,theorganizationsdevelopwrittenagreementsfortrainingandothersupportfromTPA(i.e.,the

commitmentandcontracting phase)and thenprogress to the implementationplanningphase,duringwhich an intermediary organizationmay also be involved for co-creation. Additional examples of an

integratedapproachtosupportinggenericandTripleP-specificimplementationactivitiesareprovidedin

Table1.

21

Figure5.AligningTriplePspecificandmoregenericimplementationsupport.

With attention to both Triple P-specific and generic implementation capacities, external providers of

implementationsupporttakeabuilding-blockapproachacrossstagesofimplementationtostrengthen

individualandorganizationalabilitiesfortheeffectiveuseofTripleP.Ofcourse,thoughstagesarehelpfulforconceptualizingtheimplementationprocess, implementationandscale-uparewidelyrecognizedasdynamic,nonlinearprocessesinvolvingmultipledecisions,notasingleeventthatoccursovertime.TriplePAmericaandintermediaryorganizationsneedtobeproficientathandlingthecomplexentanglementof

naturalimplementationprocesses.

22

IMPLEMENTATIONSTAGES12,27andEXAMPLEACTIVITIES

Exploration Installation InitialImplementation FullImplementation

GenericImplementationActivities

• Assessingcommunitywellbeingneeds

• Assessingsystemreadinesstoimplementchange

• Assessingcurrentsystemimplementationcapacity,andplanningtostrengthengapsandmanagechallenges

• Settingupleadershipandimplementationteams

• Professionaldevelopmenttouseandsupportactiveimplementationstrategies

• Communitycoalitioncapacitydevelopmentofimplementationinfrastructure(e.g.,practitionerselection,training,coachingsystems;localqualityandoutcomemonitoringsystems;linkingcommunicationprotocols)

• Identifyingandaddressingadaptiveimplementationchallenges

• Strengtheningcoalitionandmulti-levelsystemsenvironments

• Usingprocessandoutcomedatatoimproveoverallimplementationcapacityandperformance

• Institutionalizingoverallimplementationcapacityandperformance

• Localcoalition-regulationofongoingimplementationandprogramoptimization

• Considerationofhowtoalignoraddadditionalevidence-basedprogramsandpracticestomeetcommunitygoals

TripleP-SpecificImplementationActivities

• ClarifyingpotentialfitforTripleP(e.g.,targetpopulation,workforce)

• ClarifyingcapacitiesneededforchosenTriplePlevels,formats,andgoals

• Receivinghigh-qualityTriplePtraining

• MeetingTriplePaccreditationstandards

• EstablishingTriplePpeersupportnetworks(PASSModelthatbuildscollectiveregulation)

• FacilitatingaccessandengagementforTriplePfamilyservices

• DeliveringTriplePprogramstofamilies

• EvaluatingTriplePdeliveryandrefiningpractices

• UsingdatatoimproveorganizationalsupportforimplementingTripleP

• BuildinglinkagesacrossTriplePlevelsandorganizations

• Sustainingservicedeliveryandsupportprocesses

• Examiningandenhancingpopulation-wideimpact

Table1.ExamplesofgenericandTripleP-specificimplementationactivitiesandsupportsbystages.

23

AStage-BasedApproachtoSupportingtheScale-upofTriplePinCountiesorCommunitiesAldridge,Brown,Bumbarger,Boothroyd,andRoppolohaveproposedacoresetofpracticecomponentsforexternalimplementationsupport.1Similartotheneedtoflexiblydrawonidentifiedimplementationstrategies,93-95corepracticecomponentsofimplementationsupportmightbedifferentiallyusedacrossstages of implementation. Aldridge et al.’s arrangement of practice components by stage ofimplementation support is presented in Figure 6.27 When core practice components are flexibly yetintentionallyusedovertime,externalimplementationsupportoffersgradualandongoingcontributionstostrengthenlocalimplementationcapacityandperformancewhileensuringlocalsystemownershipoftheprocess.

AlthoughTriplePAmerica’simplementationconsultantsutilizemanyofthesecorepracticecomponents,in the sections that followwe discuss the necessary incorporation of these practice components forexternal implementation support within the stage-based activities of intermediary organizations andcommunity Triple P coalitions as they work together to scale the Triple P system. RecommendedimplementationsupporttoolsandmeasuresofformativeandsummativeimplementationoutcomesarepresentedinappendicesI-III,andwillbeincludedinseparateresourcematerialsandmadeavailableontheICTPwebsite(http://ictp.fpg.unc.edu)asfinalized.

Readiness&ExplorationCreatingongoingopportunitiesforreadinessandexplorationworkwithincommunitiesintendingtooralreadyscaling-upTriplePwasakeyrecommendationfromcountyandstatewideTriplePstakeholderswho participated in TPIE-Qualitative.51 High quality readiness and exploration processes have beenassociated with greater implementation success and efficiency in later stages of implementation,74includingforTripleP.96

Figure6.Implementationsupportcorecomponentsbystagesofimplementationsupport.1

24

Ifnotalreadycompleted,someintermediaryorganizationsmaybeabletosupportcommunityleaderswithlocalneedsassessments,whichwillgrounddiscussionsabouttheadoptionofTriplePinlocalhealthandwellbeingdataand set-upkey indicatorsofpopulation-level success thatmaybemonitoredovertime.Additionally,TriplePfitandfeasibilitywithlocalcommunityneedsandcoalitionpartnersshouldbeconsidered,collaborationbetweenco-creationpartnersmustbereinforced,localleaders’senseofchangecommitmentandchangeefficacywithTriplePneedstobegauged,andresourcesandsupportsrequiredtosustainablyscaletheTriplePsystemshouldbedetermined.ToolsandmeasuresthatcansupporttheseactivitiesarepresentedinAppendicesI-III.

Once readinesshasbeenestablishedanddecisionsmade tomove forwardwithTriplePandexternalimplementationsupportpartners,threecorepracticecomponentsforexternalimplementationsupportareessentialforintermediaryorganizationsastheysupportthecreationofalocalfoundationforTriplePsuccess: (1) build collaborative relationships, (2) assess community wellbeing goals and currentimplementation processes, and (3) facilitate collaborative agreements about implementationimprovementgoals,strategies,andearlywins.

Buildcollaborativerelationships.Thedevelopmentofcollaborativerelationshipsbetweenprovidersof external implementation support and local system stakeholders has been one of themost widelydiscussedfactorsinrelationtohighqualityimplementationsupport.17,57-59,62-64,70,92,97-99Inparticular,KatzandWandersmanpropose seven relationship characteristics that are importantbetweenprovidersofexternal implementation support and support recipients: trust, respect, collaboration, adjusting toreadiness,strengths-based,autonomy-supportive,andrapport.58

IntermediaryorganizationsneedtoproactivelyfostersuchrelationshipcharacteristicswithcommunityTripleP coalitionsand their co-creationpartners. Inpart, this canbe facilitatedearlyby collaborativeconversations about, and assessments of, community prevention goals and strengths and needs ofcoalitionimplementationpractices.Duringtheseconversations,intermediaryorganizationscanreinforceand build on existing strengths and be transparent about how they can, and cannot, be helpful tocommunityTriplePcoalitions.Wherecoalitionsmayhaveneedswithwhichtheintermediaryorganizationcannot help, brokering connectionswith new co-creation partners or resources can add value to therelationship.57 Additionally, social interactions between collaborative partners, such as periodic socialmeetings and other events that bring people together (often around food), are cited as essentialintangiblesnecessarytonurtureconnectionsonwhichsocialchangeeffortsmayrely.13

Assesscommunitywellbeinggoalsandcurrentimplementationprocesses.Collectingdataaboutlocal goals and capacity allows providers of external implementation support to accommodatecommunities’needsandresources inawaythatrecognizescurrentprogressandenablesastrengths-based approach.58 During exploration-stage assessment activities, several discussion protocols andassessmentinstrumentsrelatedtolocalimplementationcapacityandperformancemaybehelpfulandare presented in Appendices I-III. The importance of using specific assessments of need to tailorimplementationsupportandimplementationplanning,ratherthanrelyingonlyonglobalassessments,hasbeendocumented in thecontextofadvancingcommunity-widepreventionefforts.98 It is stronglyrecommended that intermediary organizations and coalition leaders incorporate specific measures ofimplementationcapacityandperformanceandnotrelysolelyondiscussiontoolsorotherglobalinquiryprotocols.

25

Facilitatecollaborativeagreementsaboutimplementationimprovementgoals,strategies,andearly wins. In preparation for the next stages of implementation, Triple P America, intermediaryorganizations, and community Triple P coalition leaders will benefit from the development of localimplementationimprovementplans,groundedinidentifiedcommunitystrengthsandneedsandguidedby strategic, evidence-informed strategies for implementation and scale-up. Because of the technicalnatureof theseplans and the likelybenefitsof community TripleP coalition leaders tobe supportedduringtheirdevelopment,intermediaryorganizationsandTriplePAmerica,workinginpartnership,mayfacilitatethegenerativeprocess.LocalimplementationimprovementplansmayincludedetailsrelatedtoseveralfeaturesoftheICTPIntegratedTheoryofChange(seeFigure1,above),suchas:

• Targetgoalsfor,andplansformeasuringandmonitoring,localpopulation-leveloutcomes;

• Target goals for, and plans for measuring and monitoring, Triple P system implementationoutcomes(e.g.,fidelity,reach,accessibility,systemalignment);

• Plans for the development or improvement of local implementation capacity and the localimplementationperformanceneededtomeettargetgoalsforTriplePdeliveryandpopulation-leveloutcomes(i.e.,linkingleadershipandimplementationteamswithinthecommunitycoalitionandtheiralignmentwithinthestate’smulti-levelsystemofimplementationsupport;workforcedevelopmentinfrastructure;qualityandoutcomemonitoringsystems;andmediaandnetworkingcapacity);and

• Plans for improving the involvementofco-creationpartners to supportandparticipate in thedevelopmentoflocalimplementationcapacityandoverallcoalitionsustainability.

Implementation improvement plans should establish a clear, direct connection between the Triple Psystem’sunderlyinglogicmodelortheoryofchangeandthebenchmarksthatsignifyhigh-qualityTripleP delivery across a community. Furthermore, several supporting implementation, practice, and policyresources (e.g.,MOUs,data-sharingplans,peersupportnetworkplans, fidelitymonitoringplans, localTriplePStayPositivemediaplans)maybeacquiredandadaptedfromstatewideTriplePpartnersormayotherwiseneedtobedevelopedasaresultoflocalimplementationimprovementplans.

LocalimplementationimprovementplansalsoallowintermediaryorganizationstodeveloptheirthoughtsforresponsivelysupportingcommunityTriplePcoalitions.Inthisway,externalimplementationsupportplans can likewise be developed, informed by identified strengths and needs in communityimplementation capacity and locally established implementation goals. Through discussion withcommunityTriplePleaders,intermediaryorganizationscanselectaseriesofprofessionalandcoalitioncapacity development strategies that may best fit local team structures, needs, and preferences forexternalimplementationsupport.

Asa corepartof the implementation improvementprocess, thecommunityTripleP implementationteamshouldbeformallyidentified.ThisteamisoftentheongoingpointofconnectionforintermediaryorganizationsandTriplePAmericaImplementationConsultantstosupportTriplePimplementationandscale-upinthecommunity.CommunityTriplePimplementationteams,oftenledbyoneortwoTriplePcoordinators,areresponsibleforday-to-daysupportofthecommunityTriplePcoalitionanditsmemberserviceagencies.Amongotheractivities,atleastaportionoftheteammayassureday-to-dayfunctionsforactiveimplementationandscale-upatthecoalition-level,including:8

26

• Assessingandcreatingongoingbuy-inandreadinesswithinandacrossTriplePserviceagenciesandcommunitypreventionsystems;

• Installing, ensuring the aligned operation of, and sustaining cross-coalition implementationinfrastructureandbestpracticestosupportTriplePdelivery;

• Developingandimplementingcoalition-levelactionplanstomanagestage-basedwork;

• Ensuringtheuseofdata,includingTriplePfidelityandoutcomedata,withinserviceagenciesandacrossthecommunityTriplePcoalitionforcontinuousqualityandoutcomeimprovement;

• Involving key partners and community members, including the children and parents beingserved,inTriplePimplementationsupportactivitiesanddecision-makingforcommunitysystemimprovement;

• Organizinganddirectingtheday-to-dayflowofinformationacrossthecoalitionand,asneeded,tostatewidepartnerstosupportlocalTriplePimplementationandscaling;and

• Identifying and addressing coalition implementation barriers and ensuring the spread ofsolutionsacrossthecoalitiontosupportsuccessfulTriplePimplementationandscaling.

Althoughimplementationteamstructuresmayvaryaccordingtolocalcontextandresources,attheircore,implementationteamsmaybenefitfrom:

• Beingreal,organizationallyrecognizedteams;15

• Havingatleastthreemembers(thoughobservationsfromTPIEsuggestthatcommunitycoalitionTriplePimplementationteamsmaybenefitfromasmanyasfourtosevenmemberswith3.0+FTEdedicatedacrosstheteamincludingatleastoneortwofulltimecoordinators);8,12,27,50,100and

• Havingthefollowingexperiencesandembeddedteamcompetencies:8,12

o experiencecreatingandmanagingsystemschangestosupporttheimplementationandscale-upofaninnovation;

o confidenceusingdatatomakeday-to-daydecisionsand improvements insupportofinnovation;

o proficiencywithTriplePandqualitybenchmarksforTriplePimplementationandscale-upacrossacommunitycoalition;and

o proficiencywiththeuseofevidence-informed,activeimplementationstrategies.

Embeddingalocalevaluatorordatamanagerwithinthecoalitionimplementationsupportteammaybeofbenefit,andisbeingdoneinTriplePcoalitionsinsomestates.

Morebroadly,thecommunityTriplePimplementationteammayworkinpartnershipwithcommunityTriplePcoalitionleadership,whichshouldalsobeclearlyidentifiedasacorepartoftheimplementationplanningprocess,tosupportcollectiveimpactbackbonefunctionsonbehalfofthecommunityTriplePcoalition,including:13

• Providingoverallstrategicdirection,

• Facilitatingdialoguebetweenpartners,

• Managingdatacollectionandanalysis,

27

• Handlingcommunications,

• Coordinatingcommunityoutreach,and

• Mobilizingfunding.

Observations from TPIE evaluation results and feedback from cross-system partners during TPIE-Qualitative greatly suggested the importance of strong community Triple P coalition leadership andimplementationteamsforlocalsuccess.50,51

Inconclusion,itshouldberecognizedthatalthoughallco-creationpartnershavearoleinsupportingthetime,space,andresourcesneededforarobustexplorationprocess,intermediaryorganizationsmaybeuniquely positioned to reinforce these activities and, in doing so, createmore effective and efficientengagementprocessesforotherco-creationpartnersinvolved.

CapacityDevelopmentFollowingtheexplorationprocess,intermediaryorganizationspartnerwithcommunityTriplePcoalitionleaders and implementation teams to strengthen local implementation capacity through strategiesmutually established during the exploration process. Two core practice components for externalimplementationsupportmaybeessential for intermediaryorganizationsduring this stage: (1)provideadultlearningforlocalleadersandteams(implementationscienceandbestpractice),and(2)facilitatelocalimplementationcapacitydevelopment(usingPDSAtechniquesasappropriate).

Provide adult learning for local leaders and teams (implementation science and best practice).Intermediary organizations contribute to the professional development of local Triple P leaders andimplementation teammembers so they can confidently support the implementation and scale-up ofTriplePusingactiveimplementationstrategies.Recentevidencesuggeststhatwhencommunitycoalitionmembersbetterunderstandmodelsofevidence-basedprogramsupportwithincommunitycoalitions,they may better support evidence-based program delivery with fidelity.10 Professional developmentneedsmayvarybetweencommunityTriplePcoalitionleadersandimplementationteams.Forexample:

• Community Triple P coalition leaders may need to reinforce their adaptive leadership skills,strengthen resources for engaging community members in local implementation activities,developunderstandingofcommonbarriersandfacilitatorstothesuccessfulscale-upofTripleP,andhaveaclearunderstandingofhowtoensurethatcommunityTriplePcoalitionpoliciesandpracticesareinalignmentwithevidence-informedimplementationpractices.

• Community Triple P implementation team members may need to develop skills related toincreasing coalition partners’ readiness and buy-in, know the intricacies of installingimplementation infrastructure across the community coalition, have proficiency in Triple Pinterventionsandactiveimplementationstrategies,andbeskilledinmanagingactionplansandlocalevaluationandimprovementsystems.

Inaddition,needsmayvarywithingroupsof localTripleP leadersor implementationteammembers,necessitatingadaptiveprofessionaldevelopmentstrategiesthatsupportarangeofpriorexperienceandknowledge.

Tosupportprofessionaldevelopmentinactiveimplementationforindividuals,intermediaryorganizationsmay draw from the broad array of implementation science frameworks available in the research orprofessional literature.However, sticking tooneor two frameworks forconsistencyofmessagingand

28

terminologymayfacilitatelearnerdevelopment.ICTPprojectslargelyrefertotheActiveImplementationFrameworks12,27andtheirrelatedliterature,aswellastotheliteratureaboutcommunity-widescale-upofevidence-basedpreventionprograms(e.g.,Communities thatCare,PROSPER,GettingtoOutcomes,andCollectiveImpact).InadditiontotherelevanceoftheseliteraturestotheTriplePImplementationFramework,86theseliteraturebasesgreatlyinformedthedevelopmentoftheICTPIntegratedTheoryofChangeandwillinformICTPtoolandresourcedevelopment.

Regardlessoftheliteratureorframeworkschosen,intermediarieshavearesponsibilitytoensuretheirlocal usability. This means that implementation science knowledge and skills should be teachable,learnable,doable,repeatable,andassessableinpractice90aswellaslocallyresponsiveandrelevant.

Facilitatelocalimplementationcapacitydevelopment(usingPDSAtechniquesasappropriate).AcrossthefourareasoflocalimplementationcapacitydescribedintheICTPIntegratedTheoryofChange(referbacktoFigure1),intermediaryorganizationsmaycontributetothedevelopmentoforganizationaland team structures, resources and abilities, and policies and practices to support implementation.Successfulcapacitydevelopmentcanbeguided,whereappropriate,throughtheuseofPlan-Do-Study-Act (PDSA) and other continuous quality improvement cycle techniques.57,61,65,76,101 IntermediaryorganizationsmayhelpTriplePcoalitionimplementationteamsdesignandimplementPDSAcyclesandmayprovidecoachingonthePDSAprocessduringcapacitydevelopmentactivities,thoughaccountabilityforlearningandproblemsolvingneedstoresidewithinthelocalcoalitionitselftosupportlocalownershipofprogress.

Leadership & Implementation Teams. As discussed throughout this plan, leadership andimplementation team structures, linked within and across levels of community service systems, areconsideredandhavebeendemonstratedtobekeyfeaturesofevidence-basedprogramimplementationand scale-up.2,4,6-20 To link leaders and teams across community service systems, the formation ofcommunity coalition structures, which may utilize lead or backbone organizations to support well-defined local serviceagencycollaboratives that share resourcesandaddresscommongoals,hasbeenutilized as a key strategy.2,14,19,20,102,103 Recently, the emergenceof the collective impact literature hasofferedkeyprinciplesforcross-sectorcommunitycollaborations.13,104-106Coalition-basedapproachestothescale-upofTriplePmayalsoprovideagoodwaytoensureopportunitiesforcross-agencyinteractionandsupport,whichwassuggestedasanimportantfactorforsuccessfullysupportingthescale-upofTriplePbycross-systemrespondentsinTPIE-Qualitative.51

Intermediaryorganizations,inconsultationwithTriplePAmericaasneeded,mayworkwithcommunityTriplePcoalitionleadersandimplementationteamstocontributetothedesignanddocumentationofcoalitionprinciplesandpractices,supportthedevelopmentofbackboneorganizationcapacity13,105withintheleadagency,anddevelopandutilizecriteriafortheselectionoflocalserviceagenciestoparticipateintheTriplePcoalition.

BeyondtheorganizationandexpansionofcoalitionstructurestosupportcommunitywideTriplePscale-up,formalizingTriplePcoalitionleadershipandimplementationteamsidentifiedduringthereadinessand exploration stage, organizing Triple P service agency leadership and implementation teams asagencies join the coalition, and linking these teams together across the coalition structure may beimportantduringthisstageofsupport.12,14,17,27TPIEresultsindicatedthatmorefullydevelopedandlinkedleadershipandimplementationteamswithinTriplePserviceagenciesweresignificantlyassociatedwith

29

agency continuation of Triple P implementation during the two-year evaluation period.50 Tools andmeasuresrelevanttotheseactivitiesaredetailedinAppendicesI-III.

Workforcedevelopmentinfrastructure.ProfessionaldevelopmentinfrastructuretosupportcoalitionTriplePpractitionerstosustainablydeliverTriplePinterventionswithfidelityandappropriateflexibilityincludesthefollowing:5,8,27

• Practitioner recruitment and selection infrastructure. Intermediary organizations may workalongside Triple P America and coalition leadership and implementation teams to establishpoliciesandpracticesforrecruitingand/orselectingcommunityservicepractitionerstodeliverTriplePprograms.PractitionerselectioncriteriamayvaryacrossTriplePprograms.Regardless,formalizingclearcoalitionpoliciesandpracticesthatintegrateimplementationbestpracticesforthis core implementation componentmay supportmore successful and sustainable selectionoutcomes.

Results from TPIE indicated that selecting only one Triple P practitioner within an agencydramatically increased the risk that the agencywould not continue to support Triple P overtime.50AgenciesthatcontinuedimplementationacrossTPIE’sevaluationperiodhad,onaverage,overthreeTriplePpractitioners.Thissortofclusteringofpractitionersisalsoreflectedinotherimplementation science literature.100 These findings did not pertain to independent or solopractitioners (often therapists in privatepractice). Regardless, coalitionpolicies andpracticesmightreflectclusteringinanefforttoensurethatasufficientnumberofTriplePpractitionersare selected and maintained within service agencies to support sustainment of Triple Pimplementation.

• Practitioner Triple P training infrastructure. Triple P America plays a strong role in trainingcommunityservicepractitioners todeliverTripleP.However, intermediaryorganizationsmayworkalongsideTriplePAmericaandTriplePcoalitionleadershipandimplementationteamstoestablishcoalitionpoliciesandpracticesthatalignwith,andreinforce,TriplePAmerica’strainingpractices.

• TriplePpractitionercoachinginfrastructure.TripleP’smodelofongoingpractitionercoachingfollowingTriplePaccreditationrequires laying infrastructure forcoalitionand/oragencypeersupportnetworks.TripleP’sPeerAssistedSupervisionandSupport(PASS)modeloffersprinciplesandpracticestoensurethatpeersupportnetworksaresufficienttoexpectintendedcoachingoutcomesforTriplePpractitioners.

IntermediaryorganizationsmayworkalongsideTriplePAmericaandcoalition leadershipandimplementation teams to lay the infrastructure forTriplePpeer supportnetworks.Alternatecoaching infrastructure that integrates implementation best practices might need to beconsidered inadditiontoor insteadofpeersupportnetworks if feasibilityorappropriatenessconcernsarisewiththePASSmodel.

Results from TPIE suggested that infrastructure to support Triple P practitioners’ ongoingcoachingfollowingaccreditationwastheareamostinneedofdevelopmentacrossparticipatingcounties.50Practitioners’ongoingreceiptofcoachingfollowingaccreditationmaybeparticularlyimportanttosustainthereachofTriplePinterventionswithinacommunity(byincreasingthelikelihoodthatpractitionerswillactuallydeliverTripleP)andtosupportthedeliveryofTriplePinterventionswithfidelity.22,26,29,32,107

30

Qualityandoutcomemonitoringsystems.QualityandoutcomemonitoringsystemstosupportimplementationandpracticeimprovementacrosscommunityTriplePcoalitionsincludethefollowing:5,8,27

• Fidelityassessmentinfrastructure.Thedeliveryofprogramswithfidelityhasconsistentlybeenlinkedtoincreasedlikelihoodofprogramoutcomes.2,80,81,108,109However,themeasurementandachievementofprogramfidelityinappliedsettingshasoftenbeenchallenging.36,41,110,111EvidencefromTPIEdoesnotsuggestotherwise: results indicatedthat infrastructuretosupportTriplePfidelityassessmentwasinparticularneedofdevelopmentacrossparticipatingcounties.50

Intermediary organizations may work alongside Triple P America, Triple P researchers andprogramdevelopers,andTriplePcoalitionleadershipandimplementationteamstoensuretheavailabilityofpracticalfidelityassessmentinstrumentsforuseincommunityservicesettingsandto establish coalition policies and practices that align with, and reinforce, Triple P America’squalityassurancepractices.

• Decisionsupportdatasystems.Usingdatatocontinuouslyimproveimplementationandprogramdelivery may lead to higher quality services, greater likelihood of intended outcomes, andprogram sustainability.37,40,112 Diagnostic and evaluative capacity is a necessary component ofengaging in data-based improvement activities.42 Although results from TPIE indicated thatdecisionsupportdatasysteminfrastructurewasinstrongshapeatcoalition-levelsinparticipatingcounties (likely due to requirements for participation in the state Triple P evaluation), resultssimultaneously suggested that additional development of decision-support data systeminfrastructurewasneededacrossTriplePserviceagencies.50

IntermediaryorganizationsmayworkalongsideTriplePAmericaandTriplePcoalitionstoidentifykeydataconstructsthatwillbeimportantfordecision-makingandperformanceimprovement;developpracticaldatacollection,analysis,andreportingprotocols;developpoliciesandpracticesthatreinforcetheuseofdataamongleadershipandimplementationteamsfordecision-makingandimprovement;andensurethatdatacoalitionandevaluationprocessesalignwithstateTripleP evaluation and funder requirements. Consideration of both implementation outcomes andpopulation-leveloutcomes,asdescribedinthefinalelementsoftheICTPIntegratedTheoryofChange(refertoFigure1,above),maybeimportant.

• Leadershipand implementationteams&practice-policycommunicationcycles.Althoughthedevelopmentofleadershipandimplementationteamshasalreadybeendiscussed,itisimportanttonotetheirrelevancetotwoothercoreimplementationcomponentsforqualityandoutcomeimprovement:facilitativeadministrationandsystemsintervention.5,27Facilitativeadministrationpracticesrelatetotheuseofinformationaboutagency/coalitionpolicyandpracticefacilitatorsandbarrierstoimprovetheimplementationofTripleP.SystemsinterventionpracticesrelatetotheuseofinformationaboutTriplePsuccessesandlargersystemsneedstoimproveandsustaintheimplementationofTripleP.Inthisway,theyareflipsidesofthesamecoin:changinginternalpoliciesandpractices(facilitativeadministration)vs.influencingexternalenvironmentalcontextsand external systems policies and practices (systems intervention). Linking leadership andimplementationteamstogetherwithfront-linepractitionersandwithexternalpolicymakerstocreate practice-policy communication cycles is an important part of developing capacity forfacilitativeadministrationandsystemsinterventionpractices.

Although results from TPIE indicated that facilitative administration and systems interventioninfrastructurewasingood-to-strongshapeatcoalition-levelsinparticipatingcounties(possibly

31

aided by coalition-based approaches), results simultaneously suggested that additionaldevelopmentoffacilitativeadministrationandsystemsinterventioninfrastructurewasneededacrossTriplePserviceagencies.50IntermediaryorganizationsmayworkalongsideTriplePAmericaand coalition leadership and implementation teams to ensure that infrastructure and bestpracticesforfacilitativeadministrationandsystemsinterventionareembeddedatalllevels.

Media and networking capacity. As previously mentioned, developing media and networkingstrategies tomobilizeknowledgeandbehaviorchangeappears tobean important factor inachievingpopulation-level outcomes.14,43-49 The Triple P system is unique in that it offers Stay Positive mediastrategiesthatcanbeadopted,combinedintoalocalmediacampaign,andstrategicallydeployedwithincommunitysocialnetworksto:

• Increasethevisibility,accessibility,andreachofTriplePinthelocalcommunity;

• Offernormativeinformationaboutchilddevelopment;

• Destigmatizetheneedforparentingsupport;and

• Introducesociallearningandmodelingopportunitiesintothecommunityatscale.

IntermediaryorganizationsmayworkalongsideTriplePAmerica,statewideStayPositivemediasupports,andTriplePcoalitionleadershipandimplementationteamstostrategicallydevelopandimplementalocalStay Positive media campaign based on community preferences, demography, geography, socialnetworks,andothercharacteristics.

SocialnetworkinganalysistechniquescanbehelpfulinmappingthesocialnetworksofcommunityTriplePcoalitionmemberstoinformstrategicplacementofStayPositivemediastrategiesandaccelerateword-of-mouth dissemination of Triple P information. Valente and colleagues49 describe and provide somesimplemeasuresofsocialnetworksthatcanbeusedtomonitorandimprovesocialnetworksinthelocalcommunity.

InconclusionandacrossallareasofcommunityTriplePcoalitioncapacitydevelopment,itisimportanttonotethatpartnershipsandregularcommunicationwiththeNorthCarolinaTriplePLearningCollaborativeandstatewidefunderswillhelpensurethealignmentoflocalimplementationcapacityandmeasurementeffortswithstatewideTriplePsystemactivities.

SupportedPerformanceAscommunityTriplePcoalitions’implementationcapacitiesarestrengthenedandtheybegintoapplytheirresourcesandabilitiestodeliverTriplePprogramsacrossthecommunity,intermediaryorganizationsmayservecoalitionswellbycloselysupportinginitialperformanceefforts.Infact,theinitialimplementationstageisoftenreferredtoasthe“awkwardstage”becausenewsystembehaviorsoftencomeintoconflictwithlongstandingsystemhabitsandadaptivechallengesbecomefullyapparentinthemidstofthepushtoperform.101Threecorepracticecomponentsforexternalimplementationsupportmaybeessentialforintermediaryorganizationsduringthisstage:(1)facilitateexperientialandinhibitorylearningopportunitieswithinthesystem,(2)providesupportivebehavioralcoachingforleadersandteams,and(3)facilitatecollectivelearningandadaptiveproblemsolving.Morethananyotherpracticecomponentsofimplementationsupport,thesethreemustworkincloseconcert.

32

Facilitateexperientialandinhibitorylearningopportunitieswithinthesystem.Duringthestageof supported performance, it becomes important for community Triple P coalition leaders andimplementation teams to fully test out their new resources and abilities to support Triple Pimplementationandscale-up.Thetendencyforcoalitionleadersandmemberstofallbackintofamiliar,longstanding, and often insufficient implementation habits and fragmented systems practices can begreat,particularlyascomfortwithnewpracticesandpartnersremainsemergent.Butleaningonnewlyorganizedcoalitionresourcesandabilitiesastheadaptiveworkofimplementationandscale-upescalatescanprovideessentialopportunitiesforindividualandcollectivelearning.Theselearningopportunitiesandcollectiverisk-taking,eveninthefaceofanxietythatmayhavebeenlearnedduringprioreffortstocreatechange, are essential to identifying areas of required improvement and sufficiently sustaining systemchange.57,113Intermediaryorganizationsneedtohelpmotivatestafffromalllevelsofthecoalition–andco-creationpartners–tofullyleanintothediscomfortofnewwaysofworkthatwereestablishedduringthecapacitydevelopmentstageofimplementationsupport.

Provide supportive behavioral coaching for leaders and teams. As community Triple P coalitionleadersandimplementationteamsfullyengageinthepracticeofTriplePimplementationandscale-up,intermediaryorganizationscansupportcompetentimplementationpracticesandnurtureconfidencebyprovidingsupportivebehavioralcoaching.63,70,97Intermediaryorganizationstakeontheresponsibilityforensuring that implementationandscale-upworkmoves forwardwithout thecoalitionor itsmembersbecomingoverwhelmedorlosingtheircollectivesenseofefficacy.101

Chilenskiandcolleagues70describetheuseof“anempowermentapproachthat includesaskingopen-endedquestionswhichencourage[teamleadersandmembers]tocriticallyreflectontheirknowledgeandexperience,encouragingteamstobrainstormprosandconswhenassessingsolutionstothemanychallengesthattheywillface,providingpositiveyetconstructivefeedback,helpingtheteamleaderandteamsetrealisticgoals,andencouragingteam-centeredaccountability”(p.26).Eiraldiandcolleagues97note the importance of directly observing local implementation processes to provide personalizedfeedbackandproblemsolving.Similarly,Rayandcolleagues61foundthaton-sitecoachingwasparticularlyimportant, as external providers of implementation support had difficulty achieving skill change andimprovement via phone or email communications. This core practice component for externalimplementation support is a parallel process to coaching practitioners’ delivery of new front-lineprograms and practices, which has been found to increase the use and quality of innovationdelivery.22,26,28-34

Facilitatecollective learningandadaptiveproblemsolving. Finally, intermediaryorganizationscansupportcommunityTriplePcoalitionleadersandimplementationteamsinthisstageofimplementationsupportbyfacilitatingcollectivelearningandadaptiveproblemsolving.17,57,63,97,114Collectivelearningmayinvolvetakingregularopportunitiestostepbackfromimplementationimprovementactivitiestoreflectonwhatstrategieshavebeenworking,why,andforwhom.Alternatively,collective learningmaybearesultofiterativedecisionsmade,actionstaken,andcorrectiveprocessestakentoensurethatstrategiesaresucceeding.Adaptiveleadershipandproblem-solvingapproacheshavebeendiscussedabove.TripleP America Implementation Consultants also may play an important, program-specific role in theseactivities, since barriers and challenges may arise related to either the Triple P system or to theimplementationstrategiesbeingusedtosupporttheTriplePsystem.

33

LocalRegulationAscommunityTriplePcoalitions’implementationperformancestabilizesandintendedimplementationandprogramoutcomesbegintoappear,intermediaryorganizationsmaybegintoconsidertaperingtheirsupport.Twocorepracticecomponentsmaybeessentialforintermediaryorganizationsduringthisstage:(1)reinforcelocal-regulationofimplementationprocessesand(2)transitiontheimplementationsupportrole.

Reinforcelocal-regulationofimplementationprocesses.Duringthisstageofimplementationsupport,coalitionleadersandimplementationteamsshouldbeexpressingadecreasedneedforintensivesupportfrom intermediary organizations and be ready to locallymanage the continual improvement of theirimplementationstructuresandprocesses.SandersandMazzucchelli115detailfivekeyelementsofparentself-regulation hypothesized to be built through Triple P interventions: self-management tools, self-efficacy, personal agency, self-sufficiency, and problem-solving. Here we reframe these principles ascoalition-regulationprinciplesthatcanbenurturedbyintermediaryorganizationsthroughoutallstagesoftheirengagementwithcommunityTriplePcoalitions,butshouldbeparticularlyreinforcedduringthislaststage:

• Collective-management tools: During earlier stages of implementation support, intermediaryorganizationsmayhavecontributedtothedevelopmentofcoalitionteamstructures,protocols(e.g., professional development plans, data plans, communication protocols), measures, andothertoolstosupport localmanagementofTripleP implementationandscale-up. Inthisfinalstageofsupport,intermediaryorganizationsneedtoreinforcetheongoingintegrationanduseofthesecollective-managementtoolsbycoalitionleadersandimplementationteams.Intermediaryorganizations should also encourage coalition leaders and implementation teams to regularlyupdate related documents, such as team terms of reference, in the case of system or staffchanges.

• Collective-efficacy: Mentioned earlier in relation to the practice principle local ownership ofprogress, collective-efficacy represents coalition leaders’ and implementation teammembers’perceptionsof their collectiveabilities tousenew implementation structuresandpractices toattaindesiredimplementationoutcomes.Reinforcingcoalitionleaders’andimplementationteammembers’ sense of collective-efficacy during this final stage of implementation support canenable them to confidentlywork together to continue tomakeprogress and improvement incommunitypreventionandwellbeingefforts.

• Collective agency: Intermediary organizations may also reinforce coalition leaders’ andimplementation teams’ sense of collective agency in determining local Triple P goals andimprovinglocalimplementationcapacityandperformancetoreachthosegoals.Thisensuresthatcoalitionleadersandimplementationteamstakeresponsibilityfor,feelownershipof,andhaveinfluenceovertheactionsthatsupportcoalitionchangesandimplementationpractices.Italsofacilitatesintermediaryorganizations’transitionawayfromanintensivesupportrole.

• Collective-sufficiency: Reinforcing coalition leaders’ and implementation teams’ collective-sufficiency during this stage does not mean increasing their isolation from support. Instead,coachingduringthistimecanfocusonreinforcingcoalitionleaders’andimplementationteams’membershipwithinlargersupportnetworks,andenablethemtoindependentlysolveproblemswithongoingsupportfromintermediaryorganizationsandotherco-creationpartnersasneeded.

34

• Adaptive problem solving: By building adaptive leadership skills with coalition leaders andimplementation team members, intermediary organizations inherently reinforce adaptiveproblemsolving,whichbydefinitioninvolvesgivingworkbacktopeoplecollectivelyinsteadofdriving decision-making from above.72 Ongoing adaptive leadership and problem-solvingcapabilitiesmaybenecessary ingredients for thesustainabilityofEBPsdue to the fluidityandcomplexityofcommunitysystemsenvironments.

Transition the implementation support role. If intermediary organizations take too great of aresponsibilityforensuringlocalimplementationprocesses,fadingtheirsupportrolecanbedifficultandalsotransferringmoreimplementationleadershiptocoalitionleadersandimplementationteamshasagreaterlikelihoodoffailing.70,116,117Insomesituations,itmaybehelpfulforintermediaryorganizationstodevelop an explicit transition strategy with community Triple P coalitions. In other situations, therealizationthatcoalitionleadersandimplementationteamshavestartedcoalition-regulationprocessessignals the transitionof the intermediaryorganizationoutof theregular flowof local implementationwork.

Therewilllikelybefuturecircumstancesthatcreatevulnerabilityforsustainingeffectiveimplementationand scale-upwithin communityTripleP coalitions– timesof turbulenceandchange incommunityorstatewideenvironments,changesinleadership,andchangesinthefitorfeasibilityoftheTriplePsystem.Intermediaryorganizationscanbeproactivebydiscussingthesepotentialswithcoalitionleadershipandimplementationteamspriortotransitioningawayfromlocalimplementationprocessesandexplorehowthese circumstances might be addressed should they arise. There are occasions when re-engagingintermediaryorganizationsmaybeparticularlyappropriateandofferaconstructiveapproach.

Finally, the collaborative partnership andmutual learning that has takenplace between intermediaryorganizations, community Triple P coalition members, and other co-creation partners should berecognized and celebrated. Because intensive relationships between external providers ofimplementationsupportandlocalpartnersarelikelytolastacrossseveralyears,ensuringspaceandtimeforhealthyreflectionandcelebrationcanstrengthenthepartnershipevenasittakesanew,lessintensiveform.Thisalsopromotesthelikelihoodthatcoalitionmemberswillcontinuetoreachouttointermediaryorganizationsforongoingneedsorsharethebenefitsofengagingwiththeintermediaryorganizationwithstatewidecolleagues.

ACoreStoryofImplementationSupport,FlexiblyAppliedInsummary,ICTPprojectssuggestthatthedevelopmentofimplementationcapacity,tailoredwithinlocalcontexts, may be most effectively served by the common-elements approach detailed through thisimplementationsupportplanratherthanbyaprescriptiveapproachtoexternalimplementationsupport.Becausetheprocessofimplementationoccursincomplexanddynamicenvironmentsanddependsonlocalfactorssuchasresources,stability,andtiming,implementationsupportactivitiesmayneedtobeadaptedthroughoutthesupportperiod.

ICTPprojectsthereforeofferacorestoryofimplementationsupportthatcanbeflexiblyappliedacrosscommunity Triple P coalitions. As depicted in Figure 6, intermediary organizations adaptively provideimplementationsupportbyresponsivelyintegratingpracticeprinciplesandcorepracticecomponentstotailorthesupportprocess,whichhasbeenwidelydiscussedasakeyfactorforsuccessfullycontributingtolocalimplementationcapacityandperformance.57-59,61-65,101,114,117,118

35

Figure6.Unpackingtheexternalimplementationsupportprocessforcontributingtothedevelopmentoflocalimplementationcapacityandperformance.

36

APPENDIXI:RecommendedToolstoSupportImplementation&Scale-UpProcesses

Duringtheimplementationandscale-upprocess,severaltoolsmaybeusefultocommunityTriplePcoalitionleadersandimplementationteamsastheydeveloplocalimplementationpoliciesandpractices.Inaddition,externalprovidersofimplementationsupportmayfindthesetoolshelpfulastheyworkcloselywithcoalitionleadersandteamstosupportlocaldecision-makinganddocumentation.

ToolName BriefDescription UsedbyRecommended

StageorFrequencyLink

TheHexagon

Tool

TheNationalImplementationResearchNetwork’sHexagonToolisareviewanddiscussiontoolorganizedaroundsix-components(need,fit,resources,evidence,readiness,capacity)thatcanbeusedtosupportprogramselection.ThistoolcanbeusedbothforinitialdiscussionsabouttheadoptionoftheTriplePsystemandforongoingdiscussionsabouttheadoptionofadditionalTriplePprogramsandmediastrategieswithinthesystem.

Coalitionleaders

Readiness&Exploration;asneededwhenconsideringadditionalTriplePprograms

http://implementation.fpg.unc.edu/resources/hexagon-tool-exploring-context

ICTPReadiness

Worksheet

Series

Aseriesofreadinessworksheets,adaptedfromRomneyandcolleagues’TriplePreadinessworksheets,96canbeusedtopreparelocalstakeholdersforunderstandingandmanagingexpectationsforimplementingorscalingTripleP.TheseworksheetscanassistlocalstakeholderstoconsiderrequirementsandpracticesassociatedwitheffectiveTriplePimplementationandidentifygapsthatmayneedtobeaddressedbeforemovingforward.

(1) Community Triple P Readiness forScaling Triple P: thisworksheet can be

Intermediaryorganizations;Coalitionleaders&implementationteams;Serviceagencyleaders(particularlytheServiceAgencyversion)

Readiness&Exploration;CapacityDevelopment

DraftworksheetseriesisbeingdevelopedbyICTPprojects

37

used to determine the presence ofcommunitystructuresandpracticesthatmay indicate level of readiness tosupport communitywide Triple P scale-up.

(2) CommunityTriplePCoalitionReadinessto Participate in Intermediary Support:this worksheet allows intermediaryorganizations to collect basicinformation about community Triple Pcoalitions and to assess key factorsrelatedtoreadinessforpartneringwithanintermediaryorganizationforTriplePimplementationsupport.

(3) Service Agency Readiness forImplementingTriplePInterventions:thisworksheetcanbeusedtodeterminethepresenceofkeyserviceagencypracticesthatmay indicate level of readiness tosupport implementation of Triple Pinterventions.

ICTP

Implementation

Capacity

DiscussionTool

Series

Thisseriesoffivesemi-structureddiscussionprotocolscanbeusedtofacilitatecollaborativeinquiryandinformalassessmentoftheinvolvementofco-creationpartnersandthefourcoreareasofimplementationcapacity:leadershipandimplementationteams,workforcedevelopmentinfrastructure,quality

Intermediaryorganizations;Coalitionleaders&implementationteams

Readiness&Exploration;asotherwisehelpfulduringlaterstages

https://ictp.fpg.unc.edu/sites/ictp.fpg.unc.edu/files/resources/Community%20Implementation%20Capacity%20ExplorationTool_Facilitator%20Copy%2004.30.18.pdf

38

andoutcomemonitoringsystems,andmediaandnetworkingcapacity.

Theseprotocolsmaybeparticularlyhelpfulwhenmorestructuredorspecificimplementationassessmentsareunwarranted,astheywerewrittenaboutglobaldetailsoflocalwellbeinggoals,coalitionstrategicplans,coalitionpoliciesandpractices,andimplementationsuccessesandneeds.

Despitetheutilityofthesediscussiontools,theimportanceofusingspecificassessmentsofneedtotailorimplementationsupportandimplementationplanning,ratherthanrelyingonlyonglobalassessments,hasbeendocumentedinthecontextofprogressingcommunitywidepreventionefforts.98Therefore,itisstronglyrecommendedthatcoalitionleadersandintermediaryorganizationsnotrelyonlyonthesesemi-structureddiscussiontools.

Technical

Assistance

Analysis

DiscussionTool

Blase118offersaquantitativeassessmentofrequiredintensityoftechnicalassistancethatcanbeusedtoscoretenrelevantfactorsonacontinuumfrombasictointensivetechnicalassistance.Thisdiscussiontoolcanhelpdiscerntheintensityofexternalimplementationsupportneededtomatchthedegreeofchangebeingundertakenbyalocalcommunity.Blaseacknowledgestheneedformoreintensiveimplementationsupportandchangefacilitationwhenthereisasignificant

Coalitionleaders,intermediaryorganizations,andco-creationpartners

Readiness&Exploration

http://challengingbehavior.fmhi.usf.edu/do/resources/documents/roadmap_4.pdf(seepage4)

39

discrepancybetweencurrentanddesiredpractice.

CreatingTeam

Termsof

Reference

Worksheet

Thisworksheetcanbeusedtofacilitatediscussionaroundseveralpossiblecomponentsofteamtermsofreference,includingteamobjectives,functions,characteristics,membershipandroles,communications,anddatause.Discussionofsomeorallofthesecomponentscanprepareateamtocreateamoreformaltermsofreferencedocument.

IndividualteamswithincommunityTriplePcoalitions(e.g.,coalitionleadershipteams,coalitionimplementationteams,agencyleadershipteams,agencyimplementationteams)

CapacityDevelopment

DraftworksheethasbeendevelopedbytheICTPprojects.

Communication

Protocol

Worksheet

Thisworksheet,developedbytheNationalImplementationResearchNetwork,helpsteamswithinanagencyoracrossagenciesestablishnewcommunicationpatternswithclearexpectationsandroles.Beyondlinkingleadershipandimplementationteams,thesecommunicationprotocolscanalsobeestablishedbetweengroupsoffront-linepractitionersandagencyorcoalitionleadershiptosupportpractice-policycommunicationcycles.

LinkedteamsorgroupswithincommunityTriplePcoalitions

CapacityDevelopment

SeeLesson9:http://implementation.fpg.unc.edu/modules-and-lessons#

ICTPLocusof

Responsibility

Worksheet

Clearlyestablishingresponsibilityforvariousaspectsofimplementationcanbechallenginginthecontextofcommunitycoalitionsandmulti-levelsystemsofimplementation

Coalitionleadershipteams;Coalition

CapacityBuilding DraftworksheetisbeingdevelopedbyICTPprojects

40

support.8ThisworksheetallowsintermediaryorganizationsandcommunityTriplePcoalitionmemberstodiscussandclarifythreetypesofresponsibilityacrossallimplementationcorecomponents:5,27whodoesit,whoassessesit,andwhoensuresit?VariousaspectsofresponsibilitycanbeassignedtothestatewideTriplePsupports(i.e.,theNorthCarolinaTriplePStateLeadershipTeam,NorthCarolinaTriplePLearningCollaborative,NorthCarolinaTriplePEvaluation),TriplePAmerica,CommunityTriplePCoalition,LocalTriplePServiceAgencies,LocalTriplePPractitioners,ordesignatedas“unclearandinneedoffurtherreview.”

implementationteams

TriplePAmerica

Training

Outcome

Reports

TriplePAmericaprovidestrainingoutcomereportsthatdetailpre-andpost-traininglearneroutcomesandparticipantexperiencedata(e.g.,satisfaction)foreachTriplePtrainingcourseconducted.Alsoincludedareparticipants’accreditationstatuses.Thesereportsprovidevaluableinformationforcoalitions,serviceagencies,andpractitionersontheTriplePtrainingprocess.

Coalitionimplementationteams;Agencyleadersandimplementationteams

CapacityBuilding ProvidedbyTriplePAmerica

TriplePPASS

ChecklistThischecklist,availablefromTriplePAmerica,allowstrackingofkeypeersupportactivitiesandmonitoringcorepeersupportcomponents(e.g.,useofaudioorvideoduringcasepresentations).

Coalitionimplementationteams;Agencyimplementationteams;CoalitionTriplePpractitioners

CapacityBuilding;SupportedPerformance

TriplePPASSChecklist

41

StayPositive

Media

Strategies

StayPositivemediastrategiesincludeTippapers,informationalmaterials(flyers,brochures,andposters),newspaperarticles,roadsidebillboards,televisionandradiospots,andTipSheets.Individualmediastrategiescanbeadoptedandcombinedintoalocalmediacampaign,andstrategicallydeployedwithincommunitysocialnetworks.

Coalitionleadershipteams;Coalitionimplementationteams

CapacityDevelopment

TriplePAmerica

Collective

Learning

Database

Thiselectronicdatabaseallowsthedocumentationofidentifiedimplementationbarriersandfacilitators,thestrategiesusedtoaddressbarriers,andothercollectivelearninginsightsasappropriateacrosstheimplementationandscale-upinitiative.

Coalitionleaders,implementationteams,agencyrepresentatives,andotherpartners

Acrossallstages IndevelopmentbyICTPprojects

42

APPENDIXII:RecommendedMeasuresofImplementation&Scale-Up

Severalprocessandoutcomemeasuresmaybeusefulascoalitionimplementationteamsmonitorimplementationandscale-up.Externalprovidersofimplementationsupportmayworkcloselywithcoalitionimplementationteamstoutilizethesemeasuresandmayberesponsibleforfacilitatingtheseassessmentsasnoted,needed,orhelpful.

MeasureName BriefDescription RespondentsRecommended

StageorFrequencyLink

Community

ReadinessScale

Chilenskiandcolleagues’119communityreadinessscaleoffersa15-item,four-factorassessmentofcommunityreadinesstoimplementlarge-scalecommunitychangeprojectsthatinvolveseveralcommunitypartners.Subscalesofferinformationaboutcommunityattachment,communityinitiative,communityefficacy,andcommunityleadership.

Mixedsamplesofcommunityleaders,serviceagencyrepresentatives,parents,andyouth

Duringinitialreadinessactivities

CommunityReadinessScale

CommunityTriple

PBuy-InScale

ThisscalewasadaptedbytheICTPprojectsfromPerkinsandcolleagues’120five-itemmeasureofcommunitybuy-inforlocalPROSPERinitiatives.ThescalecontainsfiveitemsthatassessthedegreetowhichinfluentialcommunityleadersarecommittedtoandchampionthecommunityTriplePinitiative.

CoalitionandagencyTriplePcoordinatorsandimplementationteammembers(excludingleadagencydirectorsandserviceagencydirectors)

Acrossallstages CommunityTriplePBuy-inScale

Organizational

Readinessfor

Implementing

Change(ORIC)

Measures

Sheaandcolleagues’10-itemORICmeasure121hasbeenadaptedbytheICTPprojectstoofferbriefmeasuresforassessingreadinesstoimplementorscaleTripleP.Theinstrumentprovidesscoresrelatedto

Coalitionleaders(CoalitionVersion);Serviceagencyleaders

Readiness&Exploration(CoalitionVersion);CapacityDevelopment

ORIC-LeadAgenciesORIC-ServiceAgencies

43

leaders’commitmenttothechangeprocess(changecommitment)andtheirsenseofefficacythatchangecanbeaccomplished(changeefficacy).

(1) CoalitionVersion:Thisversioncanbeused to assess community coalitionreadinesstoscaleTripleP.

(2) Service Agency Version: This versioncanbeused to assess service agencyreadiness to implement Triple Pinterventions. It may be helpful ascommunity service agencies areconsidered for membership in thecommunityTriplePcoalition.

(ServiceAgencyVersion)

(ServiceAgencyVersion)

TheWilder

Collaboration

FactorsInventory

This40-iteminventoryoffersaresearch-basedmeasurethatcanbeusedtoassess20collaborativefactorsamongcommunityagenciesandpartnersinvolvedinemergentorexistingcommunityTriplePcoalitions.

Co-creationpartners;Coalitionleaders,includingfrombothleadandserviceagencies

Acrossallstages TheWilderCollaborationFactorsInventory

Community

Capacity

Assessmentfor

ScalingtheTriple

PSystemof

Interventions

(CCA-TP)

TheCCA-TPwasoriginallydevelopedbyTPIEevaluatorstoprovideanassessmentofkeyabilitiesandrelatedresourcesincommunitiesimplementingtheTriplePsystemofinterventions.ForthedevelopmentoftheCCA-TP,TPIEevaluatorsreliedheavilyonpreviousassessmentprotocolsusedtomeasurethecapacityofcountiesorschooldistrictstoeffectively

Coalitionleadersandimplementationteammembers

Semi-annuallyacrossallstages

CCA-TP

44

supporttheimplementationandscalingofevidence-basedinterventions.122,123TheCCA-TPisafacilitatedgroupself-assessmentthat,havingbeenfurtherrefinedsinceTPIE,nowincludes110itemsorganizedwithinthefollowingelevenindices:

a. CoalitionLeadershipTeam,b. CoalitionImplementationTeam,c. PreventionSystemAlignment,d. ActionPlanning,e. Recruitment&Selection,f. Training,g. Coaching,h. FidelityAssessment,i. Decision-SupportDataSystem,j. FacilitativeAdministration,andk. SystemsIntervention.

Additionally,threesummaryindicescanbecalculated:

a. Coalition ImplementationTeams Index(indicesa-dabove),

b. CoalitionImplementationDriversIndex(indicese-kabove),and

c. Coalition Sustainability Planning Index(combiningthreespecificitems).

Toensurereliableassessment,theCCA-TPshouldbeadministeredbyanimplementationsupportspecialist.

Implementation

Drivers

TheIDA-TPwasoriginallydevelopedbyTPIEevaluatorstoassessthepresenceofactive

Serviceagencyleadersand

Semi-annuallyacrossallstages

IDA-TP

45

Assessmentfor

Agencies

Implementing

TripleP

Interventions

(IDA-TP)

implementationinfrastructureandbestpracticesamongTriplePserviceagenciestosupporttheintendeddeliveryofTriplePinterventions.TPIEevaluatorsreliedheavilyonpreviouslyestablishedimplementationdriversassessmentsandtechnicalassistancetoolsforthedevelopmentofIDA-TPitemsandscales.124-128TheIDA-TPisafacilitatedgroupself-assessmentthat,havingbeenfurtherrefinedsinceTPIE,nowincludes89itemsorganizedwithinthefollowingeightindices:

a. AgencyImplementationCapacity,b. Recruitment&Selection,c. Training,d. Coaching,e. FidelityAssessment,f. Decision-SupportDataSystem,g. FacilitativeAdministration,andh. SystemsIntervention.

Additionally,twosummaryindicescanbecalculated:

a. Agency Implementation Drivers Index(indicesb-habove),and

d. Agency Sustainability Planning Index(combiningthreespecificitems).

Toensurereliableassessment,theIDA-TPshouldbeadministeredbyanimplementationsupportspecialist.

implementationteammembers

46

TeamFunctioning

MeasuresChilenskiandcolleagues70utilizedfivebriefmeasurestoassesskeyaspectsofPROSPERteams’functioningintheirinvestigationoftheimportanceofcollaborationbetweenexternalprovidersofimplementationsupportandcommunitypreventionteams.

(1) Team Leadership: Chilenski andcolleagues’ eight-itemmeasure of thedegree to which team leadershipencouragesinputandconsensus,alongwith promotes a friendly work-environment, originally adapted fromKeglerandcolleagues.20

(2) TeamCulture:Chilenskiandcolleagues’eight-item measure of teamatmosphere, originally adapted fromKeglerandcolleagues.20

(3) TeamGoals:Perkinsandcolleagues’120two-item measure of the degree towhich teams have developed cleargoalsandgovernanceprocedures.

(4) Team Focus on Work: Chilenski andcolleagues’five-itemmeasureofteams’work orientation, originally adaptedfromMoos&Moos.129

(5) Team Tension: Feinberg andcolleagues’130 single-item measure ofteamtension.

IndividualteamswithincommunityTriplePcoalitions(e.g.,coalitionleadershipteams,coalitionimplementationteams,agencyleadershipteams,agencyimplementationteams)

CapacityBuilding;SupportedPerformance;LocalCoalition-Regulation

TeamLeadershipTeamCultureTeamGoalsTeamFocusonWorkTeamTension

47

TriplePService

Agency

Implementation

ClimateScale

Aseven-itemmeasureofagencyimplementationclimatewasadaptedbyTPIEevaluatorsfromKlein,Conn,andSorra’simplementationclimatescale.131Basedondatafromthismeasure,TPIEresultsindicatedthatTriplePserviceagencieswithlesshospitableimplementationclimateswereatgreaterriskfordiscontinuingTriplePimplementationduringtheTPIEevaluationperiod.50Kleinandcolleaguesdemonstratedthattheiroriginalimplementationclimatescalewasassociatedwithleadershipandmanagementsupportofinnovationimplementation.131Therefore,lowerscoresontheTriplePversionofthescalemaysuggestatimelyneedforcountyTriplePcoalitionimplementationteamstohelpre-establishserviceagencyleadershipandimplementationteams’localsupportforTripleP.

Alternatemeasuresofimplementationclimateareavailableaswell,132,133andcanbeconsideredbasedonlocalpreferencesorneeds.

TriplePserviceagencypractitionersandstaffmembers

SupportedPerformance;LocalCoalition-Regulation

TriplePServiceAgencyImplementationClimateScale

Coalition-

Regulation

Measuresofthefiveproposedcomponentsofcoalition-regulationarebeingexploredbytheICTPteam:• Collective-managementtools

• Collective-efficacy

• Collectiveagency

• Collective-sufficiency

48

• Adaptiveproblemsolving

SocialNetwork

AnalysisSocialnetworkinganalysistechniquescanbehelpfulinmappingthesocialnetworksofcommunityTriplePcoalitionmemberstoinformstrategicplacementofStayPositivemediastrategiesandaccelerateword-of-mouthdiffusionofTriplePinformation.

Coalitionleadershipteams;Coalitionimplementationteams

CapacityDevelopment

SocialNetworkAnalysis

StayPositive

MediaTracking

TriplePAmericahassuggestedseveralwaystotracktheperformanceofStayPositivemediastrategies,including:

(1) Semi-annual analytics provided byTriple P International that detailutilization of the local Stay Positivewebsite by practitioners and parents,and

(2) The Stay Positive Campaign TrackingForm, aMicrosoft Excel database thatcan be used by coalitionimplementationteamstorecorddetailsabout the number, placements,estimated reach, and intendedpurposes of Stay Positive mediastrategiesinthecommunity.

Theseareoutputtrackingmeasures–respondentsarenotapplicable

SupportedPerformance;LocalCoalition-Regulation

TriplePAmerica

TriplePSystem

Implementation

Outcomes:

Accessibility*

TomonitortheaccessibilityofTriplePinterventionswiththecommunity,TriplePcoalitionsmightsurveyparentsabouttheaccessibilityofTriplePprograms.ReviewingthegeographicdistributionofTripleP

Communityparents;Communityserviceagenciesandpractitioners

SupportedPerformance;LocalCoalition-Regulation

TriplePSystemImplementationOutcomes:Accessibility

49

practitionersacrosstheregionmayalsoprovidehelpfulinformation.

TriplePSystem

Implementation

Outcomes:

SystemAlignment*

ThePreventionSystemAlignmentIndexwithintheCoalitionCapacityAssessmentforCommunitiesScalingtheTriplePSystemofInterventions(CCA-TP)providesinformationaboutthedegreetowhichTriplePinterventionshavebeenadoptedinresponsetoidentifiedcommunitywellbeingneeds,thedegreetowhichTriplePserviceagencieshavebeenincludedtofillkeyservicegapswithinthecommunitycoalition,andtheextenttowhichcoalitionagenciesarealignedandsupportedtocollaborate.

SeeCCA-TProwabove

TriplePSystem

Implementation

Outcomes:

Feasibility*

TheTriplePFeasibilityScaleisafour-itemmeasureofTriplePimplementationfeasibility,adaptedforusewithTriplePfromWeiner,Dorsey,Stanick,Halko,Powell,&Lewis.134

TriplePcoalitionpractitioners;TriplePcoalitionpartners

SupportedPerformance;LocalCoalition-Regulation

FeasibilityofInterventionMeasure

TriplePSystem

Implementation

Outcomes:

Appropriateness*

TheTriplePAppropriatenessScaleisafour-itemmeasureofTriplePappropriateness,adaptedforusewithTriplePfromWeiner,Dorsey,Stanick,Halko,Powell,&Lewis.134ThisscalecanbeusedtomeasureeitherTriplePimplementationappropriatenesswithcommunityTriplePcoalitionpractitionersandpartnersorTriplePprogramdeliveryappropriatenesswithfamilies.

TriplePcoalitionpractitioners;TriplePcoalitionpartners;Communityfamilies

SupportedPerformance;LocalCoalition-Regulation

InterventionAppropriatenessMeasure

50

TriplePSystem

Implementation

Outcomes:

Acceptability*

Twomeasuresarerecommended:(1) TheTriplePAcceptabilityScaleisafour-

itemmeasureofTriplePacceptability,adapted for use with Triple P fromWeiner,Dorsey,Stanick,Halko,Powell,& Lewis.134 This scale can be used tomeasureeitherTriplePimplementationacceptability with community Triple Pcoalition practitioners and partners orTriplePprogramdeliveryacceptabilitywithfamilies.

(2) Triple P America’s CaregiverSatisfaction Questionnaire provides abrief measure of caregivers’satisfactionwithTriplePservicestheyhavereceived.

TriplePcoalitionpractitioners;TriplePcoalitionpartners;Communityfamilies

SupportedPerformance;LocalCoalition-Regulation

AcceptabilityofInterventionMeasure(AIM)TheCaregiverSatisfactionQuestionnaireisavailablefromTriplePAmerica

TriplePSystem

Implementation

Outcomes:

Fidelity*

ThreeapproachestomeasuringTriplePfidelityinclude:

(1) TriplePSessionChecklists:providedbyTriplePAmerica,thesechecklistsoffera session-by-session way forpractitionerstotrackandreportqualityadherencerelatedtoTriplePprogramdelivery.

(2) CaregiverEngagement:coalitionTripleP practitionersmight track and reportcaregiver engagement with Triple Pprogram activities by monitoringcaregiver participation in, and

CoalitionTriplePpractitioners

SupportedPerformance;LocalCoalition-Regulation

TriplePSessionChecklist

51

completionof, in-sessionactivitiesandbetween-sessionassignments.

(3) Dosage:coalitionTriplePpractitionersmight track and report thenumberofTriple P sessions that caregiverscomplete as a proportion of the totalnumber of Triple P sessions indicatedforagivenTriplePprogram.

TriplePSystem

Implementation

Outcomes:

Reach*

TriplePcoalitionscanmeasurereachas:

(1) thenumberofcommunityfamilieswhoreceiveTriplePinterventionscomparedto those who are eligible to receiveTriplePinterventions,and/or

(2) the number of practitioners (actively)delivering the Triple P interventionscomparedtothenumbertrained inorexpected to deliver Triple Pinterventions.

TriplePpractitionercontactrecords;TriplePpractitionertrainingrecords

SupportedPerformance;LocalCoalition-Regulation

TriplePSystemImplementationOutcomes:Reach

TriplePSystem

Implementation

Outcomes:Cost*

TherecentpublicationfromtheNationalAcademiesofSciences,Engineering,andMedicine,AdvancingthePowerofEconomicEvidencetoInformInvestmentsinChildren,Youth,andFamilies,describesandprovidesmethodsfortrackingcostofevidence-basedprogramssuchasTripleP.82

SupportedPerformance;LocalCoalition-Regulation

TriplePSystemImplementationOutcomes:Cost

TriplePSystem

Implementation

Outcomes:

Sustainability*

TriplePcoalitionscanmeasuresustainmentofTriplePservicesbytrackingtheextenttowhichcoalitionTriplePserviceagenciesandpractitionersremainactivelyimplementinganddeliveringTripleP.

Coalitionserviceagencies;CoalitionTriplePpractitioners

SupportedPerformance;LocalCoalition-Regulation

TriplePSystemImplementationOutcomes:Sustainability

52

*ThoseinterestedinbrowsingadditionalimplementationoutcomemeasuresmaybenefitfromsearchingtheSocietyforImplementationResearchCollaboration’s(SIRC)instrumentrepositoryathttps://www.societyforimplementationresearchcollaboration.org/sirc-projects/sirc-instrument-project/.AspartofastudyfundedbytheNationalInstituteofMentalHealthtoadvancemeasurementinimplementationscience,eachmeasureintherepositoryisratedaccordingtoitsevidence-baseandpragmatismforuseincommunitysettings.135,136WhileaSIRCmembershipisneededtoaccessthisrepository,interestedstakeholderscancontactICTPteammembersiftheydonothaveaccessandareunabletopurchaseone.

53

APPENDIXIII:RecommendedMeasuresandRecordsofImplementationSupportQualityThroughoutthesupportperiod,externalprovidersofimplementationsupportbenefitfromcollectingandusingdataaboutthedeliveryandoutcomesoftheirprovisionofimplementationsupporttooptimizetheircontributions.Severalmeasuresandqualityassurancetoolsmaybehelpful.

MeasureName BriefDescription RespondentsRecommendedStageor

Frequency

BriefAlliance

Inventoryfor

Implementation

Support

AdaptingMallinckrodt&Tekie’s13716-itemBriefAllianceInventory(BAI)willbeexploredbytheNCIC-TPteamtoprovideameasureofthecollaborativenatureoftherelationshipbetweenexternalprovidersofimplementationsupportandcoalitionleadersandimplementationteammembers.Initspresentform,theBAIprovidesinformationabouttworelationshipcomponents:Bonds(mutualtrust,acceptance,andconfidence),andGoals/Tasks(mutualendorsementofworkinggoalsandrelevanceofassociatedtasks).

Coalitionleadersandimplementationteammembers

Quarterlyacrossallstages

Implementation

Support

Collaboration

Scale

Chilenskiandcolleagues’70seven-itemscaletodescribethedegreetowhichthelocalteamcommunicateswithandworkscollaborativelyandeffectivelywiththeimplementationsupportteam.ThisscalewasoriginallyusedwithPROSPERPreventionCoordinators.

Membersofexternalimplementationsupportteams

Quarterlyacrossallstages

Contactwith

ExternalProviders

of

Implementation

Support

Chilenskiandcolleagues’70two-itemmeasureofthefrequencyofcontactbetweenimplementationtechnicalassistanceprovidersandcommunityteams,originallyusedwithPROSPERPreventionCoordinators.

Membersofexternalimplementationsupportteams

Quarterlyacrossallstages

Implementation

SupportFidelity

Checklists

DevelopedbyICTP,thechecklistinventoriesactivitiesessentialtodeliveryofcorecomponentsasintended.

Membersofexternalimplementationsupportteams

Throughoutallstages

54

Individual

Professional

Development

Indicators

Pre-training,post-training,andfollow-uplearningindicatorsshouldbedevelopedformajorimplementationsciencetrainingeventswithcommunityleadersandimplementationteammembers.Learningindicatorsshouldbealignedwithpre-establishedlearningobjectivesandbeappropriatetoknowledgeorskillacquisition.

Trainingparticipants,usuallycoalitionleadersandimplementationteammembers

CapacityDevelopment

Local

Implementation

Capacity&

Performance

Outcomes

Asthedevelopmentoflocalimplementationcapacityandperformancearetheprimaryoutcomesofexternalimplementationsupport,theCCA-TPandtheIDA-TPcanbeutilizedregularlytomonitorthelong-termeffectivenessofimplementationsupport.TheCCA-TP,beingadministeredtocoalitionleadersandimplementationteammembers,maybethemostappropriateanddirectmeasureoftheoutcomesofexternalimplementationsupport.Thoughrelevanttoexternalimplementationsupportoutcomesaswell,theIDA-TPmaybemorefittingtomeasuretheoutcomesofimplementationsupportfromthecoalitionimplementationteamtoTriplePserviceagencies.SeethepriorsectiononmeasuresofimplementationfordetailsabouttheCCA-TPandIDA-TP.

Implementation

Support

Performance

Monitoring

AbankofitemsandscaleshasbeenadoptedoradaptedbyNCIC-TPtoassessvariousaspectsofimplementationsupportperformance.Theseitemsandscalescanbeusedfordiscretetrainingevents,supportsessions,orsitevisits.Alternatively,theycanbeusedassummativeindicatorsofperformanceacrosstimeintervalsorentiresupportperiods.

(1) Quality item: measures participants’ attitudesaboutthequalityoftrainingorsupportdelivered.

(2) Use items: measures participants’ beliefs abouttheir future use of implementation strategiescovered during training or support events.Separately, an item is available to measureparticipants’ actual use of implementation

Trainingparticipantsandsupportrecipients,usuallycoalitionleadersandimplementationteammembers

CapacityDevelopment;SupportedPerformance;Attheendoftheexternalimplementationsupportperiod

55

strategies covered during historical training orsupportevents.

(3) Accessibility item: measures participants’ beliefsabout how easy it was to receive training orsupport.

(4) Feasibility Scale: a four-item measure ofimplementationstrategyfeasibility,adaptedfromWeiner,Dorsey,Stanick,Halko,Powell,&Lewis.134

(5) Appropriateness Scale: a four-item measure oftrainingorsupportappropriateness,adaptedfromWeiner,Dorsey,Stanick,Halko,Powell,&Lewis.134

(6) AcceptabilityScale:afour-itemmeasureoftrainingor support acceptability, adapted from Weiner,Dorsey,Stanick,Halko,Powell,&Lewis.134

56

References1. AldridgeWA,II,BrownJ,BumbargerBK.Theroleofexternalimplementationagentsin

contributingtonurturingsystemsenvironmentsforscalingeffectivepreventionstrategies.Invitedmanuscriptsubmittedforpublication.2016.

2. DurlakJA,DuPreEP.Implementationmatters:Areviewofresearchontheinfluenceofimplementationonprogramoutcomesandthefactorsaffectingimplementation.AmericanJournalofCommunityPsychology.2008;41:327-350.

3. FixsenDL,NaoomSF,BlaseKA,FriedmanRM,WallaceF.Implementationresearch:Asynthesisoftheliterature.Tampa,FL:UniversityofSouthFlorida,LouisdelaParteFloridaMentalHealthInstitute,NationalImplementationResearchNetwork.(FMHIPublicationNo.231).2005.

4. MeyersDC,DurlakJA,WandersmanA.Thequalityimplementationframework:Asynthesisofcriticalstepsintheimplementationprocess.AmericanJournalofCommunityPsychology.2012.

5. FixsenDL,BlaseKA,NaoomSF,WallaceF.Coreimplementationcomponents.ResearchonSocialWorkPractice.2009;19(5):531-540.

6. AaronsGA,EhrhartMG,FarahnakLR,HurlburtMS.Leadershipandorganizationalchangeforimplementation(LOCI):arandomizedmixedmethodpilotstudyofaleadershipandorganizationdevelopmentinterventionforevidence-basedpracticeimplementation.ImplementationScience:IS.2015;10:11.

7. AaronsGA,EhrhartMG,FarahnakLR,SklarM.AligningLeadershipAcrossSystemsandOrganizationstoDevelopaStrategicClimateforEvidence-BasedPracticeImplementation.AnnualReviewofPublicHealth.2014;35(1):255-274.

8. AldridgeWAI,BoothroydRI,FlemingWO,etal.Transformingcommunitypreventionsystemsforsustainedimpact:embeddingactiveimplementationandscalingfunctions.TranslationalBehavioralMedicine.2016;6(1):135-144.

9. BrownCH,ChamberlainP,SaldanaL,PadgettC,WangW,CrudenG.Evaluationoftwoimplementationstrategiesin51childcountypublicservicesystemsintwostates:resultsofaclusterrandomizedhead-to-headimplementationtrial.ImplementationScience.2014;9(1):134.

10. BrownLD,FeinbergME,ShapiroVB,GreenbergMT.ReciprocalRelationsbetweenCoalitionFunctioningandtheProvisionofImplementationSupport.Preventionscience:theofficialjournaloftheSocietyforPreventionResearch.2015;16(1):101-109.

11. BumbargerBK,CampbellEM.Astateagency-universitypartnershipfortranslationalresearchandthedisseminationofevidence-basedpreventionandintervention.AdministrationandPolicyinMentalHealth.2012;39(4):268-277.

12. FixsenD,BlaseK,MetzA,VanDykeM.Statewideimplementationofevidence-basedprograms.ExceptionalChildren(SpecialIssue).2013;79(2):213-230.

13. HanleybrownF,KaniaJ,KramerM.ChannelingChange:MakingCollectiveImpactWork.StanfordSocialInnovationReviewBlog:StanfordSocialInnovationReview;Jan.26,2012.

14. HawkinsJD,CatalanoRF,ArthurMW.Promotingscience-basedpreventionincommunities.AddictiveBehaviors.2002;27:951-976.

15. HigginsM,WeinerJ,YoungL.Implementationteams:Anewleverfororganizationalchange.JournalofOrganizationalBehavior.2012;33(3):366-388.

16. SaldanaL,ChamberlainP.Supportingimplementation:Theroleofcommunitydevelopmentteamstobuildinfrastructure.AmericanJournalofCommunityPsychology.2012.

17. SpothR,GreenbergM.Impactchallengesincommunityscience-with-practice:LessonsfromPROSPERontransformativepractitioner-scientistpartnershipsandpreventioninfrastructuredevelopment.AmericanJournalofCommunityPsychology.2011;48(1-2):106-119.

57

18. RhoadesBL,BumbargerBK,MooreJE.Theroleofastate-levelpreventionsupportsysteminpromotinghigh-qualityimplementationandsustainabilityofevidence-basedprograms.AmericanJournalofCommunityPsychology.2012;49(1-2):1-16.

19. KeglerMC,StecklerA,MalekSH,McLeroyK.Amultiplecasestudyofimplementationin10localProjectASSISTcoalitionsinNorthCarolina.HealthEducationResearch.1998;13(2):225-238.

20. KeglerMC,StecklerA,McleroyK,MalekSH.FactorsThatContributetoEffectiveCommunityHealthPromotionCoalitions:AStudyof10ProjectASSISTCoalitionsinNorthCarolina.HealthEducation&Behavior.1998;25(3):338-353.

21. AaronsGA,GreenAE,PalinkasLA,etal.Dynamicadaptationprocesstoimplementanevidence-basedchildmaltreatmentintervention.ImplementationScience.2012;7.

22. AaronsGA,SommerfeldDH,HechtDB,SilovskyJF,ChaffinMJ.Theimpactofevidence-basedpracticeimplementationandfidelitymonitoringonstaffturnover:evidenceforaprotectiveeffect.Journalofconsultingandclinicalpsychology.2009;77(2):270-280.

23. BeidasRS,KendallPC.TrainingTherapistsinEvidence-BasedPractice:ACriticalReviewofStudiesFromaSystems-ContextualPerspective.ClinicalPsychology:ScienceandPractice.2010;17(1):1-30.

24. DunstCJ,TrivetteCM.ModeratorsoftheEffectivenessofAdultLearningMethodPractices.JournalofSocialSciences.2012;8(2):143-148.

25. HerschellAD,KolkoDJ,BaumannBL,DavisAC.Theroleoftherapisttrainingintheimplementationofpsychosocialtreatments:Areviewandcritiquewithrecommendations.ClinicalPsychologyReview.2010;30(4):448-466.

26. JoyceB,ShowersB.StudentAchievementThroughStaffDevelopment.3rded.Alexandria,VA:AssociationforSupervisionandCurriculumDevelopment;2002.

27. MetzA,BartleyL.Activeimplementationframeworksforprogramsuccess.ZerotoThree.2012;32(4):11-18.

28. NadeemE,GleacherA,BeidasRS.Consultationasanimplementationstrategyforevidence-basedpracticesacrossmultiplecontexts:Unpackingtheblackbox.Administrationandpolicyinmentalhealth.2013;40(6):439-450.

29. SchoenwaldSK,SheidowAJ,LetourneauEJ.Towardeffectivequalityassuranceinevidence-basedpractice:Linksbetweenexpertconsultation,therapistfidelity,andchildoutcomes.JournalofClinicalChildandAdolescentPsychology.2004;33(1):94-104.

30. SchoenwaldSK,SheidowAJ,ChapmanJE.Clinicalsupervisionintreatmenttransport:effectsonadherenceandoutcomes.JConsultClinPsychol.2009;77(3):410-421.

31. StormontM,ReinkeWM,NewcomerL,MarcheseD,LewisC.CoachingTeachers’UseofSocialBehaviorInterventionstoImproveChildren’sOutcomes:AReviewoftheLiterature.JournalofPositiveBehaviorInterventions.2014.

32. Webster-StrattonCH,ReidMJ,MarsenichL.Improvingtherapistfidelityduringimplementationofevidence-basedpractices:Incredibleyearsprogram.PsychiatrServ.2014;65(6):789-795.

33. HattieJAC.Visiblelearning:Asynthesisofover800meta-analysesrelatingtoachievement.London:Routledge;2009.

34. KavanaghDJ,SpenceSH,StrongJ,WilsonJ,SturkH,CrowN.SupervisionPracticesinAlliedMentalHealth:RelationshipsofSupervisionCharacteristicstoPerceivedImpactandJobSatisfaction.MentalHealthServicesResearch.2003;5(4):187-195.

35. ChambersDA,GlasgowRE,StangeKC.Thedynamicsustainabilityframework:addressingtheparadoxofsustainmentamidongoingchange.ImplementationScience.2013;8(1):1-11.

36. HerschellAD.FidelityintheField:DevelopingInfrastructureandFine-TuningMeasurement.ClinicalPsychology:ScienceandPractice.2010;17(3):253-257.

58

37. KershnerS,FlynnS,PrinceM,PotterSC,CraftL,AltonF.UsingDatatoImproveFidelityWhenImplementingEvidence-BasedPrograms.JournalofAdolescentHealth.2014;54(3,Supplement):S29-S36.

38. KomroKA,FlayBR,BiglanA,WagenaarAC.Researchdesignissuesforevaluatingcomplexmulticomponentinterventionsinneighborhoodsandcommunities.TranslBehavMed.2016;6(1):153-159.

39. LiedgrenP,ElvhageG,EhrenbergA,KullbergC.TheUseofDecisionSupportSystemsinSocialWork:AScopingStudyLiteratureReview.Journalofevidence-informedsocialwork.2016;13(1):1-20.

40. MilatAJ,BaumanA,RedmanS.Narrativereviewofmodelsandsuccessfactorsforscalinguppublichealthinterventions.ImplementationScience.2015;10(1):113.

41. SchoenwaldSK.It’sabird,it’saplane,it’s...fidelitymeasurementintherealworld.ClinicalPsychologyScienceandPractice.2011;18:142-147.

42. WalkerSC,BumbargerBK,PhillippiJrSW.Achievingsuccessfulevidence-basedpracticeimplementationinjuvenilejustice:Theimportanceofdiagnosticandevaluativecapacity.EvaluationandProgramPlanning.2015;52:189-197.

43. DearingJW.Evolutionofdiffusionanddisseminationtheory.Journalofpublichealthmanagementandpractice:JPHMP.2008;14(2):99-108.

44. JohnsonK,QuanbeckA,MausA,GustafsonDH,DearingJW.Influencenetworksamongsubstanceabusetreatmentclinics:implicationsforthedisseminationofinnovations.TranslBehavMed.2015;5(3):260-268.

45. KhatriGR,FriedenTR.RapidDOTSexpansioninIndia.BulletinoftheWorldHealthOrganization.2002;80(6):457-463.

46. LoveSM,SandersMR,TurnerKM,etal.Socialmediaandgamification:Engagingvulnerableparentsinanonlineevidence-basedparentingprogram.Childabuse&neglect.2016;53:95-107.

47. PalinkasLA,HollowayIW,RiceE,FuentesD,WuQ,ChamberlainP.Socialnetworksandimplementationofevidence-basedpracticesinpublicyouth-servingsystems:amixed-methodsstudy.ImplementationScience.2011;6(1):113.

48. SandersMR,PrinzRJ.Usingthemassmediaasapopulationlevelstrategytostrengthenparentingskills.JournalofClinicalChildandAdolescentPsychology.2008;37(3):609-621.

49. ValenteTW,PalinkasLA,CzajaS,ChuK-H,BrownCH.SocialNetworkAnalysisforProgramImplementation.PLoSONE.2015;10(6):e0131712.

50. AldridgeWA,II,MurrayDW,PrinzRJ,VeazeyCA.Finalreportandrecommendations:TheTriplePimplementationevaluation,CabarrusandMecklenburgcounties,NC.ChapelHill,NC:FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2016.

51. AldridgeWA,II,BoothroydRI,SkinnerD,VeazeyCA,MurrayDW,PrinzRJ.QualitativeReport:TheTriplePImplementationEvaluation,CabarrusandMecklenburgCounties,NC.ChapelHill,NC:FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2016.

52. KleinKJ,ConnB,SorraJ.Implementingcomputerizedtechnology:Anorganizationalanalysis.JournalofAppliedPsychology.2001;86(5):811-824.

53. MetzA.Implementationbrief:Thepotentialofco-creationinimplementationscience.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2015.

54. MoullinJC,Sabater-HernándezD,BenrimojSI.QualitativestudyontheimplementationofprofessionalpharmacyservicesinAustraliancommunitypharmaciesusingframeworkanalysis.BMCHealthServicesResearch.2016;16(1):1-13.

59

55. MoullinJC,Sabater-HernándezD,Fernandez-LlimosF,BenrimojSI.Asystematicreviewofimplementationframeworksofinnovationsinhealthcareandresultinggenericimplementationframework.HealthResearchPolicyandSystems.2015;13(1):1-11.

56. MetzA,AlbersB.Whatdoesittake?Howfederalinitiativescansupporttheimplementationofevidence-basedprogramstoimproveoutcomesforadolescents.TheJournalofadolescenthealth:officialpublicationoftheSocietyforAdolescentMedicine.2014;54(3Suppl):S92-96.

57. BertaW,CranleyL,DearingJW,DoghertyEJ,SquiresJE,EstabrooksCA.Why(wethink)facilitationworks:Insightsfromorganizationallearningtheory.ImplementationScience.2015;10:141.

58. KatzJ,WandersmanA.Technicalassistancetoenhancepreventioncapacity:Aresearchsynthesisoftheevidencebase.PreventionScience.2016;17(4):417-428.

59. PalinkasLA,AaronsGA,ChorpitaBF,HoagwoodK,LandsverkJ,WeiszJR.Culturalexchangeandtheimplementationofevidence-basedpractices:Twocasestudies.ResearchonSocialWorkPractice.2009;19(5):602-612.

60. PowellBJ,BeidasRS,LewisCC,etal.MethodstoImprovetheSelectionandTailoringofImplementationStrategies.JBehavHealthServRes.2015.

61. RayML,WilsonMM,WandersmanA,MeyersDC,KatzJ.Usingatraining-of-trainersapproachandproactivetechnicalassistancetobringevidencebasedprogramstoscale:AnoperationalizationoftheInteractiveSystemsFramework'sSupportSystem.AmericanJournalofCommunityPsychology.2012;50(3-4):415-427.

62. RushovichBR,BartleyLH,StewardRK,BrightCL.Technicalassistance:Acomparisonbetweenprovidersandrecipients.HumanServiceOrganizations:Management,Leadership&Governance.2015;39(4):362-379.

63. StetlerC,LegroM,Rycroft-MaloneJ,etal.Roleof"externalfacilitation"inimplementationofresearchfindings:AqualitativeevaluationoffacilitationexperiencesintheVeteransHealthAdministration.ImplementationScience.2006;1(1):23.

64. WestGR,ClappSP,AverillEMD,CatesW,Jr.Definingandassessingevidencefortheeffectivenessoftechnicalassistanceinfurtheringglobalhealth.GlobalPublicHealth.2012;7(9):915-930.

65. WandersmanA,ChienVH,KatzJ.Towardanevidence-basedsystemforinnovationsupportforimplementinginnovationswithquality:Tools,training,technicalassistance,andqualityassurance/qualityimprovement.AmericanJournalofCommunityPsychology.2012;50(3-4):445-459.

66. WensingM,OxmanA,BakerR,etal.Tailoredimplementationforchronicdiseases(TICD):Aprojectprotocol.ImplementationScience.2011;6(1):103.

67. BakerR,Camosso-StefinovicJ,GilliesC,etal.Tailoredinterventionstoaddressdeterminantsofpractice.CochraneDatabaseofSystematicReviews.2015(4).

68. ChinmanM,AcostaJ,EbenerP,etal.EstablishingandEvaluatingtheKeyFunctionsofanInteractiveSystemsFrameworkUsinganAssets-GettingtoOutcomesIntervention.AmericanJournalofCommunityPsychology.2012;50(3):295-310.

69. ChilenskiSM,OlsonJR,SchulteJA,PerkinsDF,SpothR.Amulti-levelexaminationofhowtheorganizationalcontextrelatestoreadinesstoimplementpreventionandevidence-basedprogrammingincommunitysettings.EvaluationandProgramPlanning.2015;48:63-74.

70. ChilenskiSM,PerkinsDF,OlsonJ,etal.Thepowerofacollaborativerelationshipbetweentechnicalassistanceprovidersandcommunitypreventionteams:Acorrelationalandlongitudinalstudy.EvaluationandProgramPlanning.2016;54:19-29.

71. HeifetzRA,GrashowA,LinskyM.Thepracticeofadaptiveleadership:Toolsandtacticsforchangingyourorganizationandtheworld.Boston,MA:HarvardBusinessPress;2009.

60

72. HeifetzRA,LaurieDL.Theworkofleadership.HarvardBusinessReview.1997;75(1):124-134.73. MeyersDC,DurlakJA,WandersmanA.TheQualityImplementationFramework:Asynthesisof

criticalstepsintheimplementationprocess.Americanjournalofcommunitypsychology.2012;50(3-4):462-480.

74. SaldanaL,ChamberlainP,WangW,BrownHC.Predictingprogramstart-upusingthestagesofimplementationmeasure.AdministrationandPolicyinMentalHealth.2011;39:419-425.

75. AaronsGA,HurlburtM,HorwitzSM.Advancingaconceptualmodelofevidence-basedpracticeimplementationinpublicservicesectors.Administrationandpolicyinmentalhealth.2011;38(1):4.

76. ChinmanM,AcostaJ,EbenerP,MalonePS,SlaughterME.Canimplementationsupporthelpcommunity-basedsettingsbetterdeliverevidence-basedsexualhealthpromotionprograms?ArandomizedtrialofGettingToOutcomes®.ImplementationScience.2016;11(1):78.

77. PrinzRJ,SandersMR,ShapiroCJ,WhitakerDJ,LutzkerJR.Population-basedpreventionofchildmaltreatment:TheU.S.triplePsystempopulationtrial.PreventionScience.2009;10(1):1-12.

78. LeemanJ,CalancieL,HartmanMA,etal.Whatstrategiesareusedtobuildpractitioners’capacitytoimplementcommunity-basedinterventionsandaretheyeffective?:asystematicreview.ImplementationScience.2015;10(1):80.

79. ProctorE,SilmereH,RaghavanR,etal.OutcomesforImplementationResearch:ConceptualDistinctions,MeasurementChallenges,andResearchAgenda.AdministrationandPolicyinMentalHealthandMentalHealthServicesResearch.2011;38(2):65-76.

80. DaneAV,SchneiderBH.Programintegrityinprimaryandearlysecondaryprevention:Areimplementationeffectsoutofcontrol?ClinicalPsychologyReview.1998;18(1):23-45.

81. MihalicS.Theimportanceofimplementationfidelity.EmotionalandBehavioralDisordersinYouth.2004;4:83-105.

82. NationalAcademiesofSciences,Engineering,andMedicine.Advancingthepowerofeconomicevidencetoinforminvestmentsinchildren,youth,andfamilies.Washington,DC:TheNationalAcademiesPress.doi:10.17226/23481;2016.

83. WeinerBJ,LewisMA,ClauserSB,StitzenbergKB.Insearchofsynergy:Strategiesforcombininginterventionsatmultiplelevels.JournaloftheNationalCancerInstitute-Monographs.2012;2012(44):34-41.

84. WeinerB.Atheoryoforganizationalreadinessforchange.ImplementationScience.2009;4(1):67.

85. AldridgeWA,II,BlaseKA,VanDykeM,FixsenDL,MetzA.Implementingevidence-basedpreventionprograms:Fourthingspolicymakersneedtoknowwithrelatedpolicyrecommendations.ChapelHill,NC:NationalImplementationResearchNetwork,FPGChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2014,January.

86. McWilliamJ,BrownJ,SandersMR,JonesL.TheTriplePImplementationFramework:Theroleofpurveyorsintheimplementationandsustainabilityofevidence-basedprograms.PreventionScience.2016;17(5):636-645.

87. FranksRP,BoryCT.WhoSupportstheSuccessfulImplementationandSustainabilityofEvidence-BasedPractices?DefiningandUnderstandingtheRolesofIntermediaryandPurveyorOrganizations.NewDirectionsforChildandAdolescentDevelopment.2015;2015(149):41-56.

88. MettrickJ,HarburgerDS,KanaryPJ,LiemanRB,ZabelM.BuildingCross-SystemImplementationCenters:ARoadmapforStateandLocalChildServingAgenciesindevelopingCentersofExcellence(COE).Baltimore,MD:TheInstituteforInnovation&Implementation,UniversityofMaryland;2015.

89. BoothroydRI,FlintAY,LapizM,LyonsS,LoftsJarboeK,AldridgeWA,II.Activeinvolvedcommunitypartnerships:Co-creatingimplementationinfrastructureforgettingtoand

61

sustainingsocialimpact.InvitedmanuscriptsubmittedforpublicationinTranslationalBehavioralMedicine;2016.

90. BlaseK,FixsenD.Coreinterventioncomponents:Identifyingandoperationalizingwhatmakesprogramswork.In:ServicesUSDoHaH,ed.Washington,DC:OfficeoftheAssistantSecretaryforPlanningandEvaluation,OfficeofHumanServicesPolicy,U.S.DepartmentofHealthandHumanServices.;February2013:22.

91. FlaspohlerPD,MeehanC,MarasMA,KellerKE.Ready,Willing,andAble:DevelopingaSupportSystemtoPromoteImplementationofSchool-BasedPreventionPrograms.AmericanJournalofCommunityPsychology.2012;50(3):428-444.

92. MeyersDC,KatzJ,ChienV,WandersmanA,ScacciaJP,WrightA.Practicalimplementationscience:DevelopingandpilotingtheQualityImplementationTool.AmericanJournalofCommunityPsychology.2012;50(3-4):481-496.

93. PowellBJ,McMillenJC,ProctorEK,etal.Acompilationofstrategiesforimplementingclinicalinnovationsinhealthandmentalhealth.MedCareResRev.2012;69(2):123-157.

94. PowellBJ,ProctorEK,GlassJE.ASystematicReviewofStrategiesforImplementingEmpiricallySupportedMentalHealthInterventions.ResearchonSocialWorkPractice.2014;24(2):192-212.

95. PowellBJ,WaltzTJ,ChinmanMJ,etal.Arefinedcompilationofimplementationstrategies:resultsfromtheExpertRecommendationsforImplementingChange(ERIC)project.ImplementationScience.2015;10(1):21.

96. RomneyS,IsraelN,ZlatevskiD.Exploration-stageimplementationvariation:Itseffectonthecost-effectivenessofanevidence-basedparentingprogram.ZeitschriftfürPsychologie.2014;222(1):37.

97. EiraldiR,McCurdyB,KhannaM,etal.Aclusterrandomizedtrialtoevaluateexternalsupportfortheimplementationofpositivebehavioralinterventionsandsupportsbyschoolpersonnel.ImplementationScience.2014;9:12.

98. FeinbergME,RidenourTA,GreenbergMT.ThelongitudinaleffectoftechnicalassistancedosageonthefunctioningofCommunitiesThatCarepreventionboardsinPennsylvania.TheJournalofPrimaryPrevention.2008;29(2):145-165.

99. McCormackB,Rycroft-MaloneJ,DecorbyK,etal.Arealistreviewofinterventionsandstrategiestopromoteevidence-informedhealthcare:Afocusonchangeagency.ImplementationScience.2013;8:107.

100. KlestSK.Clusteringpractitionerswithinserviceorganizationsmayimproveimplementationoutcomesforevidence-basedprograms.ZeitschriftfrPsychologie.2014;222:30-36.

101. BlaseKA,FixsenDL,SimsBJ,WardCS.Implementationscience:Changinghearts,minds,behavior,andsystemstoimproveeducationaloutcomes.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2015.

102. KeglerMC,SwanDW.AnInitialAttemptatOperationalizingandTestingtheCommunityCoalitionActionTheory.HealthEducation&Behavior.2011;38(3):261-270.

103. KeglerMC,WilliamsCW,CassellCM,etal.Mobilizingcommunitiesforteenpregnancyprevention:Associationsbetweencoalitioncharacteristicsandperceivedaccomplishments.JournalofAdolescentHealth.2005;37(3,Supplement):S31-S41.

104. KaniaJ,KramerM.CollectiveImpact.StanfordSocialInnovationReview.Winter2011:36-41.http://www.ssireview.org/articles/entry/collective_impact/.AccessedMarch28,2011.

105. TurnerS,MerchantK,KaniaJ,MartinE.UnderstandingtheValueofBackboneOrganizationsinCollectiveImpact.Stanford,CA:StanfordUniversityPress;2012.

106. PhillipsD,JusterJS.Committingtocollectiveimpact:Fromvisiontoimplementation.CommunityInvestments.2014,Spring;26(1):11-17.

62

107. AaronsGA,FettesDL,FloresLE,SommerfeldDH.Evidence-basedpracticeimplementationandstaffemotionalexhaustioninchildren'sservices.BehaviourResearchandTherapy.2009;47(11):954-960.

108. MoncherFJ,PrinzRJ.Treatmentfidelityinoutcomestudies.ClinicalPsychologyReview.1991;11:247-266.

109. NaleppaMJ,CagleJG.Treatmentfidelityinsocialworkinterventionresearch:Areviewofpublishedstudies.ResearchonSocialWorkPractice.2010.

110. GottfredsonDC,GottfredsonGD.Qualityofschool-basedpreventionprograms:Resultsfromanationalsurvey.JournalofResearchinCrimeandDelinquency.2002;39(1):3-35.

111. AladjemDK,BormanKM.SummaryofFindingsfromtheNationalLongitudinalEvaluationofComprehensiveSchoolReform.Paperpresentedat:AnnualmeetingoftheAmericanEducationalResearchAssociation;April,2006;SanFrancisco,CA.

112. ChambersDA,GlasgowRE,StangeKC.Thedynamicsustainabilityframework:addressingtheparadoxofsustainmentamidongoingchange.ImplementationScience:IS.2013;8:1-11.

113. AbramowitzJS.ThePracticeofExposureTherapy:RelevanceofCognitive-BehavioralTheoryandExtinctionTheory.BehaviorTherapy.2013;44(4):548-558.

114. AmodeoM,EllisMA,SametJH.Introducingevidence-basedpracticesintosubstanceabusetreatmentusingOrganizationDevelopmentmethods.TheAmericanJournalofDrugandAlcoholAbuse.2006;32:555–560.

115. SandersMR,MazzucchelliTG.Thepromotionofself-regulationthroughparentinginterventions.ClinChildFamPsycholRev.2013;16(1):1-17.

116. GuldbrandssonK.Fromnewstoeverdayuse:Thedifficultartofimplementation.Stockholm,Sweden:SwedishNationalInstituteofPublicHealth;2008.

117. LessardS,BareilC,LalondeL,etal.Externalfacilitatorsandinterprofessionalfacilitationteams:Aqualitativestudyoftheirrolesinsupportingpracticechange.ImplementationScience.2016;11(1):97.

118. BlaseK.Technicalassistancetopromoteserviceandsystemchange.Roadmaptoeffectiveinterventionpractices#4.Tampa,FL:UniversityofSouthFlorida,TechnicalAssistanceCenteronSocialEmotionalInterventionforYoungChildren;2009.

119. ChilenskiSM,GreenbergMT,FeinbergME.Communityreadinessasamultidimensionalconstruct.JournalofCommunityPsychology.2007;35(3):347-365.

120. PerkinsDF,FeinbergME,GreenbergMT,etal.Teamfactorsthatpredicttosustainabilityindicatorsforcommunity-basedpreventionteams.EvaluationandProgramPlanning.2011;34(3):283-291.

121. SheaCM,JacobsSR,EssermanDA,BruceK,WeinerBJ.Organizationalreadinessforimplementingchange:apsychometricassessmentofanewmeasure.ImplementationScience.2014;9(7):1-15.

122. DudaMA,Ingram-WestK,TadescoM,etal.Districtcapacityassessment.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2012.

123. VanDykeMK,FlemingO,DudaMA,etal.Countycapacityassessment.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2012.

124. AldridgeWA,II,NaoomSF,BoothroydRI,PrinzRJ.Implementationdriversgroupinterviewprotocol:AssessingserviceagencyinfrastructureforimplementationofTriplePinterventions(ID-GIP-TP).ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2014,June.

63

125. BlaseKA,VanDykeMK,DudaM,FixsenDL.Implementationdriverexploration–Ananalysisanddiscussiontemplate.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2011,May.

126. BlaseK,VanDykeM,FixsenD.Implementationdrivers:assessingbestpractices.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2013.

127. OgdenT,BjørnebekkG,KjøbliJ,etal.Measurementofimplementationcomponentstenyearsafteranationwideintroductionofempiricallysupportedprograms–apilotstudy.ImplementationScience.2012;7:49.

128. VanDykeM,BlaseK,SimsB,FixsenD.Implementationdrivers:teamreviewandplanning.ChapelHill,NC:NationalImplementationResearchNetwork,FrankPorterGrahamChildDevelopmentInstitute,UniversityofNorthCarolinaatChapelHill;2013.

129. MoosRH,MoosBS.Thestaffworkplaceandthequalityandoutcomeofsubstanceabusetreatment.JournalofStudiesonAlcohol.1998;59(1):43-51.

130. FeinbergME,ChilenskiSM,GreenbergMT,SpothRL,RedmondC.CommunityandTeamMemberFactorsthatInfluencetheOperationsPhaseofLocalPreventionTeams:ThePROSPERProject.PreventionScience.2007;8(3):214-226.

131. KleinKJ,ConnAB,SorraJS.Implementingcomputerizedtechnology:Anorganizationalanalysis.JournalofAppliedPsychology.2001;86(5):811-824.

132. EhrhartMG,AaronsGA,FarahnakLR.AssessingtheorganizationalcontextforEBPimplementation:thedevelopmentandvaliditytestingoftheImplementationClimateScale(ICS).ImplementationScience.2014;9(1):157.

133. JacobsSR,WeinerBJ,BungerAC.Contextmatters:measuringimplementationclimateamongindividualsandgroups.ImplementationScience.2014;9(1):46.

134. WeinerBJ,DorseyC,StanickCF,HalkoHM,PowellBJ,LewisCC.Psychometricassessmentofthreenewlydevelopedimplementationoutcomemeasures.Oralpresentationatthe9thAnnualConferenceontheScienceofDissemination&ImplementationCo-sponsoredbyAcademyHealthandtheNationalInstitutesofHealth;2016,December;Washington,D.C.

135. LewisCC,WeinerBJ,StanickC,FischerSM.Advancingimplementationsciencethroughmeasuredevelopmentandevaluation:astudyprotocol.ImplementationScience.2015;10(1):102.

136. PowellBJ,WeinerBJ,StanickCF,HalkoHM,DorseyC,LewisCC.Towardcriteriaforpragmaticmeasurementinimplementationandmentalhealthservicesresearch.NationalInstituteofMentalHealthConferenceonMentalHealthServicesResearch;2016;Bethesda,Maryland.

137. MallinckrodtB,TekieYT.ItemresponsetheoryanalysisofWorkingAllianceInventory,revisedresponseformat,andnewBriefAllianceInventory.PsychotherapyResearch.2016;26(6):694-718.