Where am I, and who are you?Developing Research Competencies:

Charting the Course des Anges Cruser, Ph.D., MPAAssociate Professor & Administrative Director ORC

Keith Watson, DO Associate Dean for Graduate Medical EducationOUCOM/Centers for Osteopathic Research and

Education

Rorie Lee, PhD, MPH, CHESAssistant Family Practice Residency DirectorUNECOM

des Anges Cruser, Ph.D., MPA

What’s the purpose?

• Share information about research training activities

• Stimulate collaborative discussions about the need and vision for training in the conduct of research

What should I expect?KEITH WATSON

Broad perspective of trends in research training in osteopathic medical education & the relationship between evidence-based medicine and research training

NEXTEach of us will present

information about what we are doing in providing research training

10 MINUTES

KEITH WATSON Lead a discussion involving

you in identifying the conditions that constrain and those that promote the provision of research training to medical students, residents, and in post-graduation professional development venues.

Research Competency Challenges

Promoting wishes and understanding challenges

D. Keith Watson, D.O., FACOS

What is Research? (What is Research Education?)

• Do all the blind men describe the elephant in the same terms?

• Do all the players mean the same thing when referencing ‘research’ or research education?– accrediting bodies– specialty colleges– program directors– trainees– osteopathic medical schools– clinical faculty– basic science faculty– etc.)

Goals for Research Education

• Meet the program requirements for a research product

• Meet the AOA Competencies which require research

Goals for Research Education

• Develop informed consumers of research outcomes

• Equip physicians to become active participants in research trials and studies

• Prepare Osteopathic Physicians to be Project/Primary Investigators for quality research initiatives

Informed Consumer of Research Outcomes

Active Participant in Research

Studies

Project Investigator

Goals

All

Most

Some

“Research” and “Research Education”

• Evidence Based Medicine• Bench projects• Clinical trials

– Prospective?

– Retrospective?

– Randomized/controls?

• Case reports• Statistics course• Literature review• Journal Clubs

Evidence Based Medicine - Assumptions

• Volume of published material is too great to assimilate/evaluate

• Much published material is – Out of date– Not based on valid research design– Not applicable to the current patient care need under

study– Not likely to make a real difference in patient care

outcome• Most clinicians will NOT be trained or able to do

in-depth research• Clinicians need a mechanism to strategically

manage this information

Evidence Based Medicine—Current Trends

• Utilize auxiliary information services to screen literature in a reliable way from many perspectives-– Patient Oriented Evidence that Matters (POEMS) =

InfoPOEMS (InfoRetriever)– Up-to-Date– “Just in time” information at the point of care

• Critically assess literature and data ABOUT:– Diagnosis – Treatment recommendations– Laboratory/radiologic test necessity and accuracy– Pharmaceutical rep/sales information

Application of “Evidence Based Medicine” to Practice

• Research MAY define “best practice”for population under study

• Consideration of Patient Issues which demand priority against the ‘best practice’ because of individual variation/need

• Clinical judgment

ALL Must be Weighed for Ideal Patient Outcome

“Research” and “Research Education”

• Evidence Based Medicine• Bench projects• Clinical trials

– Prospective?

– Retrospective?

– Randomized/controls?

• Case reports• Statistics course• Literature review• Journal Clubs

Educational Model Assumption

• Reading and classroom work

• Volume of patients seen

• “Show me your references……”

• “See one, do one, teach one”

• etc

= Clinical CompetenceX

Educational Model Assumption

• Take a statistics course

• Do a literature review• Write a case report• Learn to read/critique

journal articles• Do a research

‘project’

= “Research Competence”X

What are the basic research competencies ALL DOs need to practice contemporary medicine?

• Be an informed consumer of research publications– Individual study question– Meta-analysis– Population vs. individual application

• Know basic statistical terminology • Know basic study design and inherent flaws• Common wisdom: “Don’t need to DO

research, just be educated about it”XXXChallenge to the concept XXX

Recommended Minimum Standard for Research Competency in OME

• End of medical school years 1-2:– History of osteopathic research– Knowledge of research vocabulary– Ability to do a literature search– Knowledge of basic statistics– Understanding of research problems uniquely

osteopathic (OMT)– Awareness of support resources available

consistent with level of competency expected

Ref: OCCTIC III Fall 2001, ECOP Fall 2001, AOA BOR Nov 2002Louisa Burns ORC March 2003

Recommended Minimum Standard for Research Competency in OME

• End of medical school years 3-4:– Ability to review and summarize journal

articles– Ability to formulate a research

question/hypothesis– Awareness of support resources available

consistent with level of competency expected

Ref: OCCTIC III Fall 2001, ECOP Fall 2001, AOA BOR Nov 2002Louisa Burns ORC March 2003

Recommended Minimum Standard for Research Competency in OME

• End of post-graduate years 1-3:– Understand the process of design and

implementation of a research project– Ability to critique journal articles– Ability to write a manuscript suitable for

publication or a grant application– Awareness of support resources available

consistent with level of competency expected

Ref: OCCTIC III Fall 2001, ECOP Fall 2001, AOA BOR Nov 2002Louisa Burns ORC March 2003

Challenges• Volume of published studies

– Emerging methods to deal with it

• Osteopathic profession’s urgency to focus on OPP research

• Unclear specialty college requirements for resident research

• Untrained (uniformed) program inspectors about research expectations

• Poor trainee writing skills• Untrained program directors (faculty mentors)• Ill defined IRB functions

Challenges• To Identify competency ‘outcomes’

that are realistic to measure– Define the appropriate achievement for

each level of the continuum

• To start research education in UGME

• To distinguish research from research education

• Fund sufficient infrastructure

Challenges

• Funding for research infrastructure– OPTI inherently disadvantaged to provide from member

dues/fees• Few sufficiently organized to obtain federal grant funds

independent of the COM (grants management issues).• Some may fund startup activity from foundations with little

chance for converting to ‘hard dollars’

– COMs not a likely source for GME trainees• Difficulty in providing faculty research resources now• Growth of grant supported infrastructure will be difficult to

expand to GME sites/programs• Conversion to hard dollar support not easy

Challenges

• Funding/Creating Research Education Infrastructure– Where/everywhere in the curriculum

continuum?– Definitions and common objective terms for

curriculum committees– Aligning agreement of curriculum committees

from:• COMS• Specialty Colleges (RESCs)• OPTIs (OGME Committees/Research Committees)

Informed Consumer of Research Outcomes

Active Participant in Research

Studies

Project Investigator

Goals

All

Most

Some

Where we were (2000)

OUCOM – UME– Summer Undergraduate Research Fellowship

(for ~ 8 OU undergraduates BEFORE acceptance to OUCOM—Eight week program)

– PMPH curriculum included research concepts– Summer Research Experience (10 weeks/~10

students) for select COM students who have finished Year 1

Where we were (2000)

CORE GME – 76 programs + 12 internships (~550 FTEs)– Multiple paper requirements from specialty colleges, struggle with

identifying uniform resources/access to help– 10/12 hospitals had functional IRBs

Some had Federal Wide Assurance, none had reciprocal agreements with Ohio University IRB

– Two RPACs had a resident paper presentation day– No RPAC or program required IRB approval of research BEFORE

the project began– CORE Research Committee had SOME seed money but proposals

coming forward were not Written well Had poor study design (if any) Had few program directors/faculty who could mentor well

Approach to GME Research

Require all hospitals to have an IRB and obtain FWA—and an Inter-Institutional agreement with OU IRB

Apply for grant funding and hire a Director for CORE Research with both statistical experience and project completion experience

Attempt to meet specialty college requirements as written for a meaningful research product for each resident

Provide a forum(s) for well conceived and completed projects to be presented

“Raise the bar” expectations at the RPAC level for program directors and trainees regarding research education and project completion

What we learned

IRB concepts are not common knowledge or high priority in community hospitals

Program directors place little emphasis and allow little time for trainee research activities– Specialty college requirements are too non-specific and give little

direction for PDs GME trainees have difficulty with writing skills

– Few know the elements of a research proposal– Few could list the usual sections of a journal article – Fewer could write a coherent project argument based on a bona-fide

literature search Few PDs or trainees understood the ‘life-cycle’ of an idea to

completed project or the steps to take/or where to turn for help Virtually ALL TRAINEES want to be involved and do a good job

with a research project….and will do so if given the tools.

Where we are (2004) OUCOM--UME

SURF Program continues for pre-admission activity

Summer Research Experience continues for select group of Yr 1-Yr 2 students

The PMPH course (old curriculum) has now been replaced in the new Clinical Presentation Continuum (CPC) with 22 contact hours of epidemiology, statistics, Evidence Based Medicine principles, and population impact presentations.– CPC is case based and small group oriented with select

lecture/problem presentation sessions.

Where we are (2004)CORE GME Research

Full time Director of CORE Research (grant funded)– Support staff = nil

All CORE Hospitals now have a functional IRB and an FWA status. Some have I-I agreements with OU IRB

Extensive faculty development programs for Program Directors have heightened research awareness and sparked support

Wide dissemination of posters/training materials/etc have helped educate and give direction.

Several forums for presentation have been created– RPAC Research Days– OOA Convention– OUCOM Research Day– Research days at CORE hospitals opened to other institution’s

resident participation…..

Where we are (2004)CORE GME Research

Resident projects are ‘guided’ through a central process of writing, IRB approval and data management– Current work in progress = ~100 active projects in the cue for

IRB and completion (up from only 4 in January of 2003) Four of eight RPACs held Senior Paper presentations

(with judged outcomes) this Spring– Case reports were not encouraged– Many well designed retrospective chart reviews were

completed– SEVERAL prospective multi-arm studies were completed and

reported with small but significant outcomes.

What we have learned

The culture must change on many fronts at the same time– One ‘Director’ can’t do it alone

GME trainees must bring research education WITH them to their residencies

Specialty colleges presently do not have realistic expectations about quality research or research education

Our trainees WANT to do quality research work and applaud our efforts…….but remain critical of the lack of infrastructure available to them.

Where we would like to go….

Integrated continuum of research education from UME through CME….

Core Research Competencies II

Scholarship and Research Training at the Residency Level

Rorie Lee, Ph.D, MPH, CHES

Elements of Successful Residency Research Environments

Define research broadly Require projects Focus on clinically-relevant research/EBM Research committee Integrate curriculum

DeHavenMJ, Wison GR, O'Connor-Kettlestrings P.Fam Med 1998:30(7): 501-7.

Elements of Successful Residency Research Environments

Program director support Time Didactics: planned curriculum, journal club Faculty involvement: mentors, research

committee, support with design/statistics Opportunities for presenting/forums

DeHavenMJ, Wison GR, O'Connor-Kettlestrings P.Fam Med 1998:30(7): 501-7.

UNECOM GME Programs

Internship Residencies in FP, NMM, Combined

FP/NMM Most but not all interns linked to our

residencies 3-year federal training grant to develop EBM

and research curriculum components for FP & Combined residencies

The Research-Oriented Curriculum at UNECOM

Incorporates elements of Evidence-Based Medicine, informatics, and research skills.

Takes broad view of research and scholarly activity, similar to Boyer’s expanded definition of scholarship*: Discovery Integration

Application Teaching

Planned links with AOA Core Competencies

*Boyer EL Scholarship reconsidered: Priorities of the professoriate. (1990)

Curriculum Components

Longitudinal curriculum Residents work with faculty mentors to

develop, propose, complete, and present research project (original or building on previous research)

Research training Faculty development

Expectations of Residents

Competence in using the medical library and evidence-based electronic resources

Awareness of the basic principles of study design, performance, analysis, and reporting.

Participation in scholarly activity including research Production of scientific paper suitable for publication

and acceptable to specialty college

Implementation

Projects begin in 2nd year of residency training

Project must be completed in 3rd year of training (4th year for Combined residents)

Residents present projects to peers and as part of Annual OPTI Research Forum

Residents encouraged to present projects at state/regional/national conferences

Implementation PGY 2

Resident chooses initial topic(s) Program Director approves topic Presents preliminary proposal (following

Outline) to Mentor group Mentor assigned once proposal approved

– Detailed study proposal prepared – Proposal submitted to IRB– Start project

Implementation PGY 3

Data collection & analysis Draft of paper submitted Paper revised

– Peer presentation– OPTI/regional/professional presentation

Final paper approved by Program Director, Review Committee & submitted to specialty college

Mentors

Residency faculty take lead role Basic science and other clinical faculty also

involved Resident progress is tracked using Checklist Time is difficult to find

What we’ve learned so far

Successive approximation: take small steps Internal feedback/review helpful Faculty & residents need accessible support Integrating EBM & clinically applied research

important in residency training Finding time is difficult but can be done Define “research” broadly: approach both

projects and people inclusively

Challenges

Expand core research mentor group & research training

Continue promoting research collaboration Involve faculty in additional scholarly

activities Sustain support once current funding

ends/obtain additional external funding

Training clinician researchers: Inside the box looking out

or outside the box looking back in?

des Anges Cruser, Ph.D., MPA

The Osteopathic Research CenterFOUNDED

January 2002

$275,000 per year for four yearsAACOM, AOA, AOF

Additional Resources Generated $2M varying duration over 2 to 4 yearsAnd Pending applications of $4.8 M

over 3 to 4 years in duration

PURPOSE

Facilitate the collective efforts of the osteopathic profession

to investigate the mechanism of action and clinical

efficacy of Osteopathic Manipulative Medicine

Advisors

Boyd Bowden, D.O.John Crosby, J.D. Leonard Finkelstein, D.O. V. James Guillory, D.O.,

MPH Felix Rogers, D.O. Michael Samuels, Dr. P.H.

Brian Degenhardt,D.O. John K. Lynch, D.O. Boyd Buser, D.O. J. Justin McCormick, Ph.D. Michael J. Patterson, Ph.D.Douglas Wood, D.O., Ph.D.Collaborative Mentors and Partners: UNTHSC, ATSU, PCOM,

UNECOM, AZCOM, WVCOM

In: Epidemiology, Physiology, Anatomy, Pulmonary Medicine….

&

SPECIFIC AIMSAim 1. Develop and sustain the infrastructure necessary to support the mission.

Aim 2. Develop OMM research training opportunities.

Aim 3. Conduct research into the mechanisms of action and clinical efficacy of OMM.

Aim 4. Facilitate inter-institutional research collaborations.

Examples of Activities• OMM Research Think Tank: establish overall direction of

the research initiatives, develop standards for research design and methodology, provide educational seminars to enhance the confidence and competence of new and established researchers.

• Research Collaboration: review and recommend procedures in blinding, placebo control, inter-rater reliability and validity, ethics, safety, OMM protocols, methodology, outcome measures, and statistical analysis

• Products: Newsletter, manuals, proposals, Focused Research Forums

.

Research Local and Inter-institutional

PregnancyCarpal Tunnel SyndromeLymphSympathetic Nervous

SystemCOPDEducationalLow Back PainFibromyalgiaPost CABG

Pneumonia

Multiple Sclerosis & Exercise

Low Back Pain

Otitis Media

NCCAM Developmental Center

Research Training

Local and National Programs

AOA Research Conference

OCCTIC

Research Fellowships in Family Medicine and OMM

Pre and Post Doctoral

Master’s of Science in Clinical Research & Education

Continuous or in three 4-month blocks

Clinical, Teaching and Research

18+ at various stages of the program

Goals and products

Research enhancement meetings

Formal coursework

Attend research conferences

Rotate through IRB

Present poster, participate in a panel discussion

Guideposts in Developing the Research Training Programs

NIH Standards

Different organizations have different rules

Human Subjects Protection

The integrity of the research design

The total human energy of research work

Role of the student or resident physician

Role of the PI, the CRC, the research team

Collaborative Osteopathic Research Training Initiative Proposal

NIH recognizes a decline in the number of successful physician PI applications

NIH strongly recommends partnerships between basic scientists and clinical researchers

NIH begins to recognize Osteopathic Medicine as a unique profession in health care

The Osteopathic Medical Community recognizes the need for organized approach to teaching research competencies

FACTORS PROMPTING THE DISCUSSION

•Graduating COM classes rate research competencies among weakest component of their medical education.

• The demands of medical school and residency programs constrain the infusion of research training

•The imperative remains.

HISTORICAL CONTEXT

October 2003: SOMA petitioned OHF for support to develop research curricula in COMs

OHF agreed to need, recommended COMs collaborate, and would consider a funding request for a collaborative commitment from COMs

Spring 2004: AACOM and the ORC approach OHF with a proposal for a National Osteopathic Research Training Initiative

NIH/NCCAM announces funding availability to support research curriculum development

ORC opens broader dialogue with the Board of Deans

Module IBASIC RESEARCH COMPETENCIES

Module IIADVANCED RESEARCH

COMPETENCIES

Module IIIMentored Clinical

Research Scientist-Practitioner

A Simple Model of Competency Based Research Training

MODULE I

A FOCUS ON THE MEDICAL STUDENT

PRIMARY AIMS:

1. Instill an appreciation for the role of research in osteopathic medicine,

2. Prepare the student for evidence based medicine practice,

3. Equip the student with basic competencies to read, interpret, discuss, and participate in clinical research.

MODULE II

A FOCUS ON THE RESIDENT OR POST GRADUATE WITH EXISTING BASIC RESEARCH COMPETENCIES

PRIMARY AIMS:

1. Takes the aspiring researcher to the next level of research activity,

2. Supports the process of developing a research proposal,

3. Guides participation or leadership in preliminary or pilot research project

Modules I and II

may be provided together in

a pre-doctoral fellowship,

a residency program

a faculty development program

METHODS

Classroom didactics

Electronic-based media for self study

Mentorship

NIH-NCCAM RFA: 15 September 2004

Year 1: Planning & Development $100,000

Years 2-4: Implement, Evaluate, Sustain $200,000/yr

AIMS:1. Increase quality and quantity of research content of curricula

2. Enhance exposure to, understanding and appreciation of

Evidenced-based biomedical research literature

and

Approaches to advancing scientific knowledge