VA Centers of Excellence in Primary Care Education: Evaluation Design and Findings

VACentersofExcellenceinPrimaryCareEduca7on:Evalua7onDesignandFindings

PhilipR.LeeIns.tuteforHealthPolicyStudiesUniversityofCalifornia,SanFranciscoJanuary11,2016AnneEeL.Gardner,PhD,MPHNancyHarada,PhD

VETERANSHEALTHADMINISTRATION 1

VETERANSHEALTHADMINISTRATION

VeteransHealthAdministra7on(VHA)StatutoryMissions

1.  HealthCare2.   HealthProfessionsTraining*3.  Researchrelatedtohealthcondi.onsofVeterans4.  EmergencyBackuptoDoD&partofNat’lDisasterMedical

System(HHS,DoD,VA,FEMA)•  VAisthelargestsingleproviderofhealthprofessionseduca.onintheUS•  124of152VAmedicalcentersand3of6independentoutpa.entclinicshave

affilia.onsforgraduatemedicaleduca.on(GME)andalmostallVAfacili.eshavesomehealthprofessionstrainees

•  VHA’seduca.onalmissionisconductedincollabora.onwith:o  130of141allopathic&22of29osteopathicmedicalschools.o  Morethan40otherhealthprofessions–representedbyover1,800colleges

anduniversi.es


ScopeofAcademicAffilia7ons

Office of Academic Affiliations * 2013 RCS 10-0161 Report

118,799TotalTrainees*40,420MedicalResidents

22,541MedicalStudents23,808NursingStudents31,380AssociatedHealthTrainees1,397DentalStudents&Residents253AdvancedFellows


CentersofExcellenceinPrimaryCareEduca7on(CoEPCE)Purpose

•  Fostertransforma.onofclinicaleduca.onbypreparinggraduatesofhealthprofessionalprogramstoworkinandleadpa.ent-centeredinterprofessionalteams;

•  Studytheimpactofnewinterprofessionaleduca.onalapproachesandmodelsonhealthprofessionseduca.onandworkforceoutcomeswithinandbeyondVA.

•  Improveprimarycareinstruc.onalstrategieswithemphasisonclinical,workplacelearning

Goalisculturechangeandincreasedunderstandingofotherprofessionsculture,values,andeduca5on

4


CoEPCESitesandAcademicAffiliates

5

Co-Directors: C.Sco/Smith,MDandMelanieNash,MSN,ANP

AcademicPartners: GonzagaUniversitySchoolofNursing

UniversityofWashingtonSchoolofMedicine IdahoStateUniversitySchoolsofPharmacyandNursing

Co-Directors:JoyceWipf,MDandKamekaBrown,PhD,APN

AcademicPartner:UniversityofWashington

SchoolsofMedicineandNursing

Co-Directors: RebeccaBrienza,MD,MPHandSusanZapatka,MSN,APN

AcademicPartners: FairfieldUniversitySchoolofNursing

QuinnipiacUniversitySchoolofNursingYaleUniversitySchoolsofMedicineandNursingUniversityofConnecScutSchoolofMedicine

Co-Directors: MimiSingh,MD,MSandMaryDolansky,PhD,RN

AcademicPartners:CaseWesternReserveUniversitySchoolofNursing

UrsulineCollegeSchoolofNursingClevelandClinicFoundaSon

Co-Directors:RebeccaShunk,MDandTerryKeane,DNP,APN

AcademicPartner:UniversityofCaliforniaatSanFrancisco

SchoolsofMedicineandNursing


•  Eachsitehasalocally-developedcurriculumthatcombines3instruc.onalstrategies:o  WorkplaceLearning(e.g.,clinical,pointofcare)o  Didac.c/FormalInstruc.on(e.g.,seminars)o  Reflec.vePrac.ce

•  Allsitesprovideinstruc.oninfourcoredomains:o  InterprofessionalCollabora.ono  SharedDecisionMakingo  SustainedRela.onshipso  PerformanceImprovement

•  TraineeslearnandworktogetherperPACTmodel•  Traineeengagementatleast30%:

o  Physicianresidentstraineeso  NursePrac..onertraineeso  Post-DoctoratePharmacyresidentso  Post-DoctoratePsychologyfellows

6


Evalua7onDesign—DocumentImplementa7onandAchievementofProgramOutcomes

7

Guiding

Principles:Service

AccountabilityDevelopmental

Partnership-based

Evalua7onQues7onsand

LogicModel:Outcomesandindicators

DataCollec7onAc7vi7es:Mixed-method

Forma.ve(process)Summa.ve(outcomesandimpacts)

VETERANSHEALTHADMINISTRATION 8

1.Transforma7vecenterInterprofessionalEduca7onandCollabora7vePrac7ce

2.Expandedpar7cipantknowledge,aTtudes,prac7ces,understandingandcompetencies

3.Increasedpar7cipantinterprofessionalcollabora7on

4.Improvedpar7cipantclinicalperformance

5.Improvedcenterpa7entoutcomes:6.Improvedcenterpar7cipantsa7sfac7on7.Improvedcenterprimarycareservices8.Improvedcenterfunc7on

9.ImprovedVAprimarycaredeliverysystem

Short-term Outcomes (2011-2012) Intermediate Outcomes (2012-2014)

Long-term Outcomes

(2015+)

CoEPCELogicModel

+ContextualAnalysis:healthcaresystem(structure,financing,accredita7on,licensure)

+Fourtosixindicatorsperoutcome


DataCollec7onAc7vi7es

ü  Semi-structuredinterviewswithtrainees(N=35,16%),faculty(N=8),CoEleadership(N=10)todescribepar7cipantexperience,facilita7ngfactorsandchallenges,andtraineesa7sfac7on(2014)

ü  Par7cipantSurveyadministeredtoPasttrainees(N=88,42%)andCurrenttrainees(N=130,82%)toassesssa7sfac7onandimpacts(2015)

ü  Semi-AnnualSiteReportsandCoEdocumenta7onofprogramdevelopment,implementa7onandsustainment(2012–2015)

ü  NursePrac77onerResidentCompetencyTool(2012–2015)ü  FiveCasestudiestodocumentsuccessfulsiteinterprofessionaleduca7on

and/orclinicalini7a7ves(2015)ü  CurriculumAnalysistodocumentCoEmodelsandtheirevolu7on(2012–

2015)ü  FeasibilityAnalysisofVAClinicalDataSets(2015)ü  FinancialAssessmentofCoEcostsandresourcessecured(2015)

9


ProgramOutcome1:TransformaSvecenterinterprofessionaleducaSonandcollaboraSve(IPEC)pracSceEvidenceofAchievement:•  Launchofinterprofessional,pa.ent-centeredmedicalhomemodelof

trainingandprovidingneweduca.ontrainingopportuni.esfortrainees•  Launchofworkplace,clinicallearningopportuni.es,e.g.,huddles,models

oflongitudinalengagement,NPResidencyprograms•  Successfultraineerecruitmentandincreaseinnumberandtypeof

professionsFacilita7ngFactors:•  CoEPCEresources,e.g.,NPs.pends•  Highlevelofpar.cipa.onbyfacultyandtraineesChallenges:•  AspectsoftheVAfacility—space,culture,protected.meforfaculty•  Traineediversity,includingschedules,clinicalexper.se


ProgramOutcome1:TransformaSvecenterinterprofessionaleducaSonandcollaboraSvepracSce

Profession AY11-12 AY12-13 AY13-14

NPResidents 3 8 12

NursingStudents 20 26 30

NPPsych-MentalHealth

1 1 2

Medicine 79 104 121

PharmacyResidents

14 17 25

PsychologyTrainees

7 8 16

Totals 124 164 206

11 *Rugen,K.W.,etal.,“VeteranAffairsCentersofExcellenceinPrimaryCareEduca.on:Transformingnurseprac..onereduca.on.,NursingOutlook.62(2014)78-88


ProgramOutcome1:CenterIPECPracSceTraineepercep7onsofachievementofCentergoalsandobjec7ves:

12

*Supportsintegra.onoftraineesinPACT*Supportsadop.onofinterprofessionaleduca.onmodels*Providespa.ent-centered,team-basedcaretoVeterans*Supportsadop.onofinterprofessionalclinicmodels

*Fosterscollabora.onwithintheVA*IsValuedbyfacilityleadership*Contributestothebodyofknowledgeofbestprac.cesforhealthprofessionseduca.on*Fosterspartnershipswithacademicaffiliates*Hascleargoalsandobjec.ves

Over40%oftraineesgavetopra.ngofeffec.venessorCompleteley

40%orlessoftraineesgavetopra.ngofeffec.veness

Majority(91.4%)sayCentershaveachievedtheirmissionto:“fostertransformaSonofclinicaleducaSonbypreparinggraduatesofhealthprofessionalprogramstoworkinandleadpaSent-centered,interprofessionalteamsthatprovidedcoordinated,longitudinalcare”


ProgramOutcomes2,3,4–ParScipantKnowledgeandPerformanceOver80%ofPastand/orCurrenttraineeslearnedandprac7cedmostofthecompetenciesinthefourcoredomains—samplefindings

13

96% 95%87% 91%

79%

95% 97% 96% 95%

79%

Learned Prac.ce%ofPastandCurrenttraineesthatsaidthey‘learned’(Yes)and‘prac.ced’askill(Agree/StronglyAgree)

Providingclinicalcareasamember

ofPACT(Currenttrainees)

Valuingtheuniquerolesandexper.seofotherhealthprofessionals

Communica.ngmyrolesandresponsibili.esclearlytoother

healthprofessionals(Currenttrainees)

Developingtrus.ngrela.onshipswithpa.entsandfamilies/caregivers

Usingperformanceimprovementstrategiestoincreasetheefficacyof

team-basedcare


81%79%

76%

84%83%

79%

84%

90%

65%

70%

75%

80%

85%

90%

95%

Collec.ngandanalyzingdatatoassessperformance

improvement

Usingperformanceimprovementstrategiesto

increasetheefficacyofteam-basedcare

Usinginforma.ontechnologytomanagepa.entpanels

Usingreflec.veprac.cetoiden.fyopportuni.esfor

improvement

Learned

Prac.ced

14

ProgramOutcomes2,3,4–ParScipantKnowledgeandPerformanceOver80%ofPastand/orCurrenttraineeslearnedandprac7cedsomeofthecompetenciesinPerformanceImprovement(PI)


47%

46%

43%

29%

29%

27%

26%

24%

21%

37%

26%

29%

32%

29%

34%

34%

23%

30%

28%

31%

22%

16%

16%

24%

24%

25%

21%

27%

28%

21%

5%

6%

7%

13%

8%

10%

19%

14%

20%

6%

Qualityoffaculty

Clinicalexperience

Interprofessionalapproachtopa7entcare

Mentoring

Programcontent

Teachingmethods

PhysicalspacewithintheCoEPCE

Alloca7onof7mebetweendidac7clearningandclinicalac7vi7es

Reflec7veprac7ce,e.g.,7meforself-reflec7on

OverallCoEPCElearningexperience

Excellent

VeryGood

Good

Fair

Poor

N/A–didnotexperience

15

ProgramOutcome6–ParScipantSaSsfacSonMajority(90%)oftraineesratedtheirCenterlearningexperiencehighlyoverall*butprogramcoulddobeierinspecificCenterdesignelements


TraineePerspec7ve

“I’vehadopportuni.estoseeallthesedifferentthingswhereIthinkalotofmypeershavenothadthesameopportuni.esorthesameopportuni.esforeduca.on.Soinasense,Ithinkitreallysetsmeapart.”(NursingTrainee)“IknowI’vetalkeduptheprogrambutIdo.Ireallyloveit.Ithinkmyeduca.onhasbenefiEedsomuch.Mypersonal,emo.onallifehasbenefiEedaswellandthatmakesmeabeEerdoctorandmoremo.vated,andbeEerreader.(MedicineTrainee)

16


ProgramOutcome8–CenterFuncSonCentersratedhighlyonmostaspectsoftraineeprepara7on.Coulddomoreintheareasofindividualassessmentandcareerdevelopment

17

*Preparestraineestoprac.cecollabora.velyinteams*Providesopportuni.esfortraineestointeractwithpreceptorsfrommul.pleprofessions*Preparestraineesforacareerinprimarycare(mapstoLPS-PC)

*Providesongoingfeedbacktotrainees*Providesadequatementoringfortrainees*Regularlyevaluatestrainees’progresstowardsmee.ngcompetencies*Providesclearexpecta.onsoftrainees*Workswithtraineestoiden.fypersonalcareergoals

Over70%oftraineesgavetopra.ngs(AgreetoStronglyAgree)

Under70%oftraineesgavetopra.ngs(AgreetoStronglyAgree)


ProgramOutcomes7and9:ImprovedVAPrimaryCareDeliverySystemIncreasedlikelihoodofseekingemploymentinprimarycare

51%58% 52% 56%

32%31%

28%28%

17% 11%20% 16%

Pasttrainees:Seekemploymentinprimarycarebeforejoining

CoEPCE

Pasttrainees:seekemploymentinprimarycareimmediatelyater

comple.ngtheCoEPCE

CurrentTrainees:seekemploymentatinprimarycarebeforejoiningtheCoEPCE

Currenttrainees:seekemploymentinprimarycare

sincejoiningtheCoE

Likely Unlikely NoDiference

18


ProgramOutcomes7and9:ImprovedVAPrimaryCareDeliverySystemIncreasedlikelihoodofseekingemploymentinVA

49%66%

50%

78%

24%13%

27%

11%16% 21% 22%

11%

Pasttrainees:seekemploymentattheVAbeforejoiningCoEPCE

Pasttrainees:seekemploymentattheVAimmediatelyatercomple.ngtheCoEPCE

CurrentTrainees:seekemploymentattheVAbefore

joiningtheCoEPCE

Currenttrainees:seekemploymentattheVAsince

joiningtheCoE

Likely Unlikely NoDiference

19


Suppor7ng,SustainingTeam-based,Pa7ent-centeredApproachtoPrimaryCare

Keyfacilita7ngfactors:ü VAsupport—local,VISN,na.onalü Adap.vetrainingmodelwithdiverseapproachestoaccommodatetraineeneeds

ü Highstakeholderpar.cipa.onü PresenceofastrongPACTü Partnerships—withinthelocalVA,acrosssites,withaffiliatesandbeyond

KeyChallenges:ü  Resourcesü  Culturethatsupportsinterprofessionaleduca.onandcollabora.ve

prac.ce

20


SummaryofFindings

•  CoEPCEissucceedingin:–  ExpandingCenteradop.onofinterprofessionalcurriculumandclinicmodels–  Expandingtraineeknowledgeandskillsinfourcoredomainsalbeittovarying

degrees–  Increasingtraineeinterprofessionalcollabora.onandateam-basedapproach

topa.entcare–  Improvingtraineeclinicalperformanceandintegra.onwithPACTteam–  Improvingtraineesa.sfac.onwithCoEPCEandmanyindividualcomponents–  ImprovingCenterprimarycareandfunc.on–  ImprovingVAprimarycaredeliverysystem

•  AreasforImprovement:–  TraineeknowledgeanduseofPerformanceImprovementskills–  Traineementoringandassessmentofprogress–  Site-levelmonitoringofspecificcurriculumcomponents,e.g.,classroom

didac.csessions

21


Lessonslearned

•  Roleofcontext—resources,leadership,localfacilitysupport

•  Opportuni.esforcross-programcollabora.on—instrumentdevelopment,buildevalua.onculture

22


Acknowledgement:Evalua7onPartners

CoEPCENa7onalCoordina7ngCenterStuartGilman,MD,MPH;DirectorKathrynRugen,PhD,FNP-BC;NursePrac77onerConsultantJudithBowen,MD;PhysicianConsultantScoiSmith,MD;PhysicianConsultantBradfordFelker,MD;PrimaryCareMentalHealthConsultantJoshuaRuberg,PhD;MentalHealthConsultantIvyTonnu-Mihara,PharmD;PharmacistConsultantLauralTraylor,MSW;ProgramManagerNancyHarada,PhD,MPA,PT;Evalua7onManagerAnneieGardner,PhD,MPH;Evalua7onConsultantDeborahLudke,MHA,Administra7veOfficerKimberlyUhl,MBA,ManagementAnalystCoEPCESitesBoiseClevelandSanFranciscoSeaileWestHaven 23

Evalua7onandAnaly7csUnit,OAADavidLa7ni,PhDJenniferHayes,EdMAnneieL.Gardner,PhDSamuelS.King,MS,MdivMichaelKashner,PhD,JD

CenterforProgramDesign&Evalua7onDartmouthCollegeRebeccaL.Butcher,MS,MPH

HealthServicesResearchMethodologyandSta7s7csCore,MichaelE.DeBakeyVAMedicalCenterJessicaA.Davila,PhDSarahMayJunYing


ForMoreInforma7on

NancyHarada,PhD,MPA,PTCOEPCENa7onalEvalua7onManagerOfficeofAcademicAffilia7onsNancy.Harada@va.gov

24