FROM KNOWLEDGE TO WISDOM IN POLICY

Stephen Birch

McMaster University

Hamilton, Ontario.

Positive analysis:

Do decision-makers use knowledge derived from scientific research?

Do scientific researchers address questions that decision-makers face?

Normative analysis:

Should decision-makers use knowledge derived from scientific research?

Should scientific researchers addresses questions that decision-makers face?

Emphasizes how to transfer not what to transfer

Driven by intellectual curiosity of researcher

Is this effective (on average in study subjects)Decision-maker’s needs

Determinants of effectivenessUnder what conditions is policy more likely to workWhat policies work under particular conditions

Conditions, circumstances, contexts

noise or confounding to researchers

substance to decision-makersBehaviour change

need knowledge on what determines behaviour and role of information on behaviour change

Is it relevant to decision-maker’s problem?Is the research question the same as the policy

question the decision-maker needs answered?

Is it understood by decision-maker?

Is it considered in decision-making process?

Is it used to determine decisions?Relationship between knowledge and behaviourRole of values

Measuring effectiveness: 5 year survivalsurviving one day or 4 years and 364 days

makes no differencesurviving 5 years and surviving 20 years makes no difference

Comparison of survival on average in two groups with similar distributions of characteristics

Relative survival among people like me?Simpson’s Paradox

Choose to study what we are paid to studyThe amazing success of the Beta blocker

Condition A Condition B

Subjects 200 200rich (n/treatment/control) 40/20/20 160/80/80poor 160/80/80 40/20/20

Outcomes (years of survival) 900 880rich 100 640poor 800 240

Effectiveness (years/n) 9 8.8rich 5 8poor 10 12

Outcome 3 year mortality post myocardial infarction

Clinical:

Propanolol 9% Placebo 12% (p<0.05)

Education:

High 6% Low 13% (p<0.05)

Stress and social isolation:

Low 2% High 14% (p<0.05)

Evidence-based guidelines: repeat cesarean section:

Clinicians survey:90% received and 85% agreed with guidelines33% would change practice, rest already in line

Clinical audit:Baseline cesarean rates higher than self reportsNo significant reduction post guidelines

Knowledge transfer focussed on effectiveness of procedure not behaviour change of provider

95% hospitals in Canada private not for profit (PNFP) but publicly funded

Provinces considering private for profit (PFP)Devereaux et al (CMAJ 2002): Effect of PFP (cf

PNFP) on patient mortalityMeta analysis (n=15) RR death PFP = 1.02“PFP results in higher risk of death”“Policy makers should take this into account”

Informing decisions or supporting ideology?All 15 studies from USAPNFP hospitals in US and Canada differ

Excludes public hospitals in USPNFP in Canada are ‘private’ in name only Purchaser-funding arrangements differ

Knowledge acquired:For a population like the US, with access to care based on ability and willingness to pay, and three distinct types of hospital ownership, for those with access to private hospitals, patient mortality is higher in for profit hospitals

KT: Lower rates of patient hospitalisation among family physicians paid capitation compared to Fee for Service (FFS)

Action: Ontario committed to shifting physician off FFS for Service to reduce hospital costs

Decision-maker needs: How does switching from FFS to capitation lower a physicians practice style?

Physician selection explained observed rates

No change in hospitalisation rates

Findings specific to levels of capitation and FFS

KT: Acheson et al. (1998) presented 39 evidence-based policies for reducing inequalities

None of the evidence reported on impact of policy on inequalities in health

Information related (at best) to impact on individual healthHealth impacts based on studies of non-poor populations

Marmot (2010) found inequalities in health haven’t reduced

KT: Cost-effectiveness ratios for new drugs - compares new drug with existing drug for same patient group

New drug costs more than existing drug so decision maker does not face choice between new and old drug

Evidence ignores opportunity cost of new drug

By calculating ratio ‘evidence’ discards information relevant to decision maker (total costs and effects)

Using ‘knowledge’ has led to increased costs and no evidence of increased health gains

KT: Structured abstract:

“Carotid endarterectomy reduced the risk of stroke”

No qualifications to this statement

One sixth of the main text devoted to the ‘special circumstances’ (highly selected patients, providers, centres) associated with the trial

Research focused on how to transfer as opposed to what to transfer

Decision-makers ought to use the information generated by scientific research

Information is just one of many determinants of behaviour Role of information in behaviour changeSo what constitutes effective KT?

Reflects the intellectual curiosity of researchers as opposed to the needs of the decision maker

What works on average in setting/population versus what works best for my setting/population