Centre for Behavioural Research and Program Evaluation

The Centre for Behavioural Research and Program Evaluation is a national program of the National Cancer Institute of Canada funded by the Canadian Cancer Society, and is located at the University of Waterloo.

Is Our Scientific Culture an Impediment to the Health of Canadians?

Roy Cameron and Barbara Riley

CPHA, Halifax

June 2, 2008

CBRPE Vision and Mission

Vision To transform cancer prevention and supportive care

systems to improve the lives of Canadians.

Mission To create knowledge, relationships and resources that

contribute to the continuous improvement of population level interventions.

Overview: In a nutshell

There is an urgent need to generate relevant evidence to guide population based chronic disease prevention intervention

Scientific organizations and dominant scientific culture are not oriented to support the science that must be done: if we do not support relevant science, are we not impeding the health of Canadians?

Scientific organizations and practices are starting to change in ways that enable generation and use of relevant evidence

Cancer21%

Cardiovascular Disease 18%

Injuries 12%

Mental Disorders

11%

All Others22%

Neurological &Sensory Disorders

9%

Chronic Respiratory Disease7%

BC ActNow: The Problem

Causes of Premature Mortality and Years Lived in Poor Health

•Source: Adapted from Evaluation of the Burden of Disease in British Columbia. Strategic Policy and Research Branch, B.C. Ministry of Health, January 2001.

100.0%

41.6%

53.6%

71.3%

27.0%27.0%

27.0%

-0.6%

16.6%

28.4%

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

04/05 05/06 06/07 07/08 08/09 09/10 10/11 11/12 12/13 13/14 14/15 15/16 16/17 17/18

Year

Per

cent

Total

Health

Education

Other

BC ActNow: The Business Case

Revenue Growth – 3%Education Growth – 3%Health Growth – 8%Balanced BudgetOther spending reaches zero by 17/18

Conceptual Framework

Coordination and StewardshipCoordination and Stewardship

AboriginalRelations &

Reconciliation

AboriginalRelations &

Reconciliation

Public Safety& Solicitor General

Public Safety& Solicitor General

Children & Family

Develop.

Children & Family

Develop.

PremierPremier

All sectors, municipalities, Non-Governmental Organizations, Agencies, Businesses and Industry

Strategic Communications

ActNow BC provides a unifying brand for the strategic cross governmental and cross-sectoral initiative for creating a healthy BC population. Facilitates “improved alignment of cross-ministry policy”

EconomicDevelopment

EconomicDevelopment

Labour &Citizen’sServices

Labour &Citizen’sServices .Environment

.EnvironmentEmployment

& IncomeAssistance

Employment & Income

Assistance

Energy, Mines &

Petroleum Res

Energy, Mines &

Petroleum Res

Small Bus. &

Revenue

Small Bus. &

RevenueAttorneyGeneral

AttorneyGeneral

Committee on Natural Resources

& the Economy

Committee on Natural Resources

& the Economy

Committee on

Social Development

Committee on

Social Development

StrongStart BCCabinet

Committee

StrongStart BCCabinet

Committee

Agriculture& Lands

Agriculture& Lands Finance

Finance

Advanced Education

Advanced Education

Forests& Range

Forests& Range Health

Health Transportation

Transportation CommunityServices

CommunityServicesEducation

Education Tourism,Sports & Arts

Tourism,Sports & Arts

Minister of StateMinister of State

Some Practical Questions

• What is the right mix of interventions to implement?

• What is most urgent and important to do given limited resources and existing assets?

• What are residents ready for?• What will work under conditions (demographic,

physical, economic, etc) in this province?• What is feasible?• How will be know that we’ve made a difference?

Answering Practical Questions

Traditional approach: Researcher-initiated studies

Example: School Health Chronology, University of Waterloo Research GroupsRCTs of smoking prevention programs (including

questions about delivery)Began to doubt generalization through time given how

rapidly youth culture evolvesProgramming also evolves; ongoing innovation and

adaptation by practitioners

Answering Practical Questions

Alternative approach: Experimentation by social actors.

Lesson from tobacco control: “Bal laughs when asked about the role of science in guiding policy decisions… ‘where there is no science you have to go and be venturesome—you can’t use the paucity of science as an excuse to do nothing…all the scientists came in behind us and analyzed what we did’” (Sweet M & Moynihan R. 2007. Improving Population Health: The Uses of Systematic Reviews. Milbank Memorial Fund and Centers for Disease Prevention and Control.)

Answering Practical Questions

“Bal is frustrated by colleagues who wait for high-level evidence before acting…. ‘Most scientists will say you need a randomized controlled trial level of proof to do a community intervention. That’s horse feathers. We tried twenty-five things—twelve worked and we renewed those. Empirical trial and error is the oldest scientific device and we used it to distinction.’” (Sweet & Moynihan, 2007)

Proposition: The most important experiments will not be done by scientists, but by social actors.

Big Implications for the Role of Science in Population Intervention

Using existing evidence: Bring to bear any relevant evidence in formulating a policy (e.g. principles of behaviour and attitude change to inform tobacco warning labels)

Generating new evidence: Studying innovative policies as they are implemented; use evaluation methodologies to study these “natural experiments” (e.g. Charlton 2004. Natural experiment. In Lewis-Beck, Bryman, Liao (eds) The Sage Encyclopedia of Social Science Research Methods (Vol 2). Sage Publications.)

Scientific Obligation: CBRPE Position

Core obligation: Study the most important questions, in the most rigorous way possible, given the conditions

Be willing to trade off rigor for relevance (Green, Glasgow…)

Foundational assumption: the scientific community has an obligation to help find solutions to urgent problems that threaten the health and social fabric of Canada; refusing to remain “pure” by doing only studies that use the most rigorous designs is not defensible given that we are funded by taxpayers and donors who depend on us.

Scientific Impediments: Examples

Structural: e.g. grant competition timelines (e.g. annual

competitions) are impediments to studying natural experiments, which are usually fleeting opportunities

Cultural, Normative: scientist tendency to equate value with rigor, and to base

peer judgment on rigor vs. relevance, potential impact lack of urgency: “some details could be improved, let

her reapply next year” – another year is 10% of the elapsed time till BC is broke

The Good News: Science is Evolving

Example #1: Methods of defining Best Practices

Best science (strength of evidence based on internal validity)

vs Most impact (promise table: Swinburn, Gill, Kumanyika, Obesity prevention: a proposed framework for translating evidence into action. Obesity reviews, 2005, 6, 23-33.)

Impact Effectiveness

Low Medium High

Low Least promising

Very promising Most promising

Medium Less promising

Promising Very promising

High Promising Less promising Promising

The Good News: Science is Evolving

Example #2: Population Health Intervention Research Initiative for Canada (PHIRIC)

Goal is to increase the quantity, quality and use of population health intervention studies

Catalyst for aligning organizational efforts to enable a ‘new’ science of population intervention

Pan-Canadian Early days

The Centre for Behavioural Research and Program Evaluation is supported by the National Cancer Institute of Canada with

funds from the Canadian Cancer Society.

We are located at the University of Waterloo.

Centre for Behavioural Research and Program Evaluation