Effectiveness Science

Marita G. Titler, PhD, RN, FAAN

Professor and Associate Dean for Clinical Scholarship & Practice

Rhetaugh G. Dumas Endowed ChairDivision III Chair

University of Michigan School of Nursing

Ann Arbor, Michigan

Objectives

Overview of effectiveness science (ES). Contributions and impact of nurse

investigators to ES Future Directions

Efficacy Studies

Investigate the benefits of an intervention under ideal and highly controlled conditions (RCTs).

Substantial differences from clinical practice Study protocol - minimize treatment preferences &

multimodal treatment programs, control of the skill levels of the person(s)

delivering the intervention, and restrictive control over the study sample.

High internal validity. Limitations in external validity (generalizability to practice)

Example: A phase III double blind trial of duloxetine to treat painful chemotherapy-induced peripheral neuropathy (Ellen Smith; ASCO meeting, 2012)

Effectiveness Studies

Examine outcomes of interventions under circumstances that approximate the real world.

Employ more heterogeneous samples, clinicians delivery of treatment (rather than a trained RA), and settings are routine clinical practice.

High external validity; more applicable to everyday practice. Sacrifice some degree of internal validity.

Examples: Impact of nursing surveillance on failure to rescue

(Shever, 2011). Dissemination of the Tobacco Tactics Intervention

In Community Hospitals (Duffy et al, 2012)

Efficacy and Effectiveness

Distinction between efficacy and effectiveness studies viewed as a spectrum along a continuum.

Study guided by State of the science on the topic. Research questions or hypotheses.

Effectiveness Science Testing healthcare interventions in real

world settings, advancing use of research findings by diverse populations, training and career development of the next generation of investigators.

Uses comparisons, subgroups and context to understand what healthcare treatments work for whom and in what settings.

(Simpson et al, 2010; special issue of Medical Care, June 2012)

Why Effectiveness Science Help consumers, clinicians, purchasers,

& policy makers to make informed decisions that will improve healthcare at the individual & population level.

Knowing efficacy does not necessarily guarantee effectiveness in practice.

Effectiveness Science includes

PCOR Comparative effectiveness research Implementation and dissemination

science Interface with electronic healthcare data

repositories, and informatics

Terminology

CER – generation and synthesis of evidence that compares alternative methods to prevent, diagnose, treat and monitor a clinical condition or improve the delivery of care.

Translation/Implementation science – testing effectiveness of implementation strategies to promote adoption of evidence in healthcare delivery.

Dissemination – targeted distribution of intervention materials to a specific public health or clinical practice audience. 

(Titler & Everett, 2002; Wensing et al 2006; NIH; IOM, 2009; Simpson, 2010)

Examples of Top Priorities CER

Primary prevention of falls in older adults Dissemination & translation strategies to promote

use of research findings Strategies to reduce HAI Comprehensive care coordination for chronic illness School-based interventions – prevention and

treatment of over-weight and obesity Strategies to prevent obesity, hypertension,

diabetes, & heart disease in at risk populations. Strategies to reduce health disparities in

cardiovascular disease, diabetes, cancer, musculoskeletal diseases, and birth outcomes

(IOM, 2009; pg 3-5)

Patient Centered Outcomes Research Institute (PCORI)

Nonprofit institute established by health care reform legislation

Assist policy makers, purchasers, consumers, and clinicians in making informed health decisions by conducting research that provides quality and relevant evidence ….

Board of governors – approved research priorities May 21, 2012.

Check out the PCORI web-site (www.PCORI.org)

Methods and Approaches

Systematic reviews Meta-analysis Practice guidelines – based on scientific

evidence Experimental designs

Pragmatic clinical trials Clustered Randomized trials

Observational Approaches Qualitative methods Mixed methods

(Conn et al, 2012; IOM, 2009; Tunis et al, 2009; Pressler, 2011)

Clustered Randomized Trials Evidence-Based Practice: From Book to Bedside

(PI: Titler; R01 HS10482). Book to Bedside: Sustaining Evidence-Based Practices in Elders (PI: Titler; R01 HS10482)

Implementation of Practice Standards for ECG Monitoring (PI: Funk; R01HL081642)

An intervention to improve outcomes in patients with advanced cancer (PI: McCorkle; R01NR011872)

RWJ foundation INQRI program – upcoming issue of Medical Care (falls; delirium; pain; substance abuse).

Incontinence Prevention: The TULIP Project (PI: Sampselle; R01NR012011; RCT)

Observational Designs

Nursing Contributions to Patient Outcomes for 3 Older Populations (PI: Titler; R01 NR05331)

Prevention of Nosocomial Infections & Cost-effectiveness Refined (P-NICER) (PI: Stone; R01NR010107).

Prevention of Nosocomial Infections & Cost-effectiveness in Nursing Homes (PI: Stone; R01NR013687)

Implementing Evidence to Prevent Urinary Infection & Enhance Patient Safety (PI: Krein; R01NR010700).

Nursing Impact on Care Outcomes For Chronically Ill & Minority Patients (PI: Aiken; R01NR004513)

Qualitative and Mixed Methods

Used in many studies Focus groups Interviews of participants Key informants

Used to understand context of care delivery Necessary to understand users’/clinicians’

perceptions about the implementation intervention, usefulness of each component, and to uncover the underlying mechanisms of the intervention.

Examples of Areas of Impact

Patient Safety and Quality ECG Monitoring (Funk et al) Fall Prevention (Titler; Anderson; Dyke) HAI (Stone, Krein, Lake)

Healthcare System Acute care (Aiken; Keenan; Titler; Lake) Long-term care (Anderson; Stone, Rantz) Ambulatory care (Friese et al)

Health Promotion (Duffy; Sampselle)

Examples of Areas of Impact

Symptom management, QOL, Palliative Care, EOL care (e.g. Daly; Northouse; Doorenbos; Dunbar; Beck; Happ; Resnick)

Cost (e.g. Stone; Titler; Berry; Dunbar; Happ)

Models and frameworks to guide the research (complexity science; implementation models; outcomes effectiveness models)

RWJ INQRI Program

Examine nurses’ practices, processes and work environments and determine the impact nurses have on the quality of patient care.

40 interdisciplinary research teams www.INQRI.org Demonstrates nursing’s unique

contributions to patients, families and communities across a diverse range of settings

(Mary Naylor and Mark Pauly)

The FOCUS Program

Designed to facilitate communication and other outcomes cancer patients and spouse/partners

Breast Cancer Clinical Trial (1996-2000) ACS

Prostate Cancer Clinical Trial (2000-2005) NCl

Advanced Cancer Clinical Trial (2005-2010) NCI

Laurel Northouse PhD, RN, FAAN

Translation, Implementation, Dissemination

Three major areas- Translation: to the web- Implementation: in community agencies

- Rosalyn Carter Institute/Cancer Support Community- Dissemination: program materials widely

National, state, local National Cancer Institute Michigan Dept of Community Health UM Comprehensive Cancer Center

Activities are occurring simultaneously

Laurel Northouse PhD, RN, FAAN

FOCUS Prostate Program Materials

Sponsored by NCI and SAMHSALaurel Northouse PhD, RN, FAAN

Evidence-Based Practice: From Book to Bedside

Improved pain assessment and pain management.

Less pain intensity. For each one-unit increase in the

Summative Index, total costs decreased by $1,598.75 (p = 0.002)

A net savings to the hospital of more than $131,000 per 100 patients, even after implementation costs are taken into account.

Improvements sustained(Titler et al, 2008 HSR; Brooks et al, 2008)

Nursing Contributions to Patient Outcomes for 3 Older Patient Populations

Purpose: to conduct outcomes effectiveness research using data in electronic repositories from one medical center. Unique contributions of nursing treatments to patient outcomes. (Exploratory study)

Sample: 4 years of data from 3 pt. populations: heart failure (1435 hosp; 1035 patients) hip fracture and procedures (569 hosp; 524

patients) those who receive Fall Prevention

intervention (10,187 hosp; 7,851 patients)

(Titler et al; Funded by NINR; R01 NR05331)

Studies and Publications

15 studies reported in peer reviewed publications Two doctoral dissertations Papers published by students during their PhD

program. Propensity scoring papers:

Qin, R., Titler, M., Shever, L., & Kim, T. (2008). Estimating causal effects of nursing intervention via propensity score analysis. Nursing Research, 57(6), 444-452.

Shever, L. L. (2011). The Impact of Nursing Surveillance on Failure to Rescue. Research and Theory for Nursing Practice, 25 (2), 127-151.

Shever, L. L., Titler, M., Kerr, P., Qin, R., Kim, T., & Picone, D. M. (2008). Effects of nursing surveillance on hospital cost for older adults at risk for falling. Journal of Nursing Scholarship, 40(2), 161-169. 

Impact Documentation of

nursing treatments to patient outcomes and cost.

Dose of nursing treatments

Explicated methods for conducting ES using large data repositories of clinical care delivery

Conceptual Model to guide research

Application of model to analytic methods (e.g. propensity scoring)

27

Implementation of Risk Specific Fall Prevention Interventions to Reduce Falls in Hospitals

Purpose - to implement, in 3 community hospitals (14 nursing units), fall reduction interventions targeted to patient specific risk factors.

Funded by RWJ foundation INQRI program

Preliminary Findings

Increased use of fall prevention interventions targeted to patient specific risk factors

Reduction in falls Rounding by nurse managers to address

fall prevention practices Request for QRGs Conduct a multi-site study

Topics: VAP reduction; Enteral tube feeding; Family presence in ICU; Depression Screening in Primary Care; Care of adults with diabetes; FacilitatingEBP during military operations.

The NNPN promotes implementation of evidence-based practices (EBP) through a collaborative model of shared learning and participation.

Multi-site research Community members

100 hospitals Represent 32 states, Puerto Rico and Singapore Hawaii State Center for Nursing

30Marita Titler PhD, RN, FAAN

Resources Education

Newsletters 14 Podcasts How to start a journal

club EBP references Eye on Evidence Webinars – lunch and

learn; journal clubs

Research Network of sites for

research Process for investigators

to access NNPN organizations for research

Organization context measurement instruments

Culture Climate Interactive human

relationshipsMarita Titler PhD, RN, FAAN

Hawaii State Center for Nursing

Hawaii Nurses Shaping Healthcare: A State-Wide Evidence-Based Practice Initiative

Legislative mandate to for EBP and quality outcomes

Debra D. Mark, RN, PhDNurse Researcher, Hawai’i State Center for Nursingdebramar@hawaii.edu

Outcomes to Date

Increasing EBP capacity across the state Trained 39 teams 8 Health care systems Institutionalizing practice change Papers and conference presentations

Future Directions

Education and training programs in ES Maximize use of registries, clinical and

administrative data repositories Multi-site research Terminology/nomenclature; taxonomy Context Measurement (e.g. self-report; MRA; observation) Complexity of the intervention – implementation

science. Core elements Mode of delivery Intervention dose (intensity and duration of each

component)

Future Directions

More collaboration with CTSAs Nurse scientists lead interdisciplinary

teams (e.g. behavioral interventions; symptom management; implementation science)

Key stakeholder partnerships at the initiation of the study.

Scale-up and spread what works Public policy messages Funding sources