Pre0174-Bhattacharyya Gouri Shankar

25.11.2016

1

Generating evidence-about effectiveness and value

Dr. G. S. Bhattacharyya

MBBS, MD, MRCP, Ph.D, DNBE

Medical Oncologist

In-Charge, Department of Medical Oncology,

Fortis Hospital, 730 Anandapur, EM By-pass Road,

Adarsha Nagar, Kolkata-700107, West Bengal, India

Member GIN. LMIC Committee Member

“Everything should be made as simple as possible but not simpler.”……………...(Albert Einstein)

The Future: Personalized Healthcare

Clinical Certainty& Efficiency

Acquisition Integration

Personalization

Rx

Diagnostic Complexity

… Sophisticated Decision Support

Diagnostics Growth

… More SpecificDiagnostics

Evidence-Based Medicine &

Patient-Centered Choice

A. Good evidence/Important to patient

B. Good evidenceC. Potential for good evidence

D. Important to patient choice/potential for good evidence

E. Important to patient choice/ No potential for evidence

A B

A. L. Cochrane, from T. Hope Evidence-based patient

choice and the doctor patient relationship in But Will

it Work Doctor? Kings Fund, London 1997, 20 – 24

CD

E

IMPORTANT

EVIDENCE

Trends In Cost Of A Twenty-Four-Week Colorectal Cancer Treatment Regimen And Change In Quality-Adjusted Cost

Of Care For Colorectal Cancer, 1998–2005.

Darius Lakdawalla et al. Health Aff 2015;34:555-561

©2015 by Project HOPE - The People-to-People Health Foundation, Inc.

Hierarchy of evidence

STUDY DESIGN

� Randomized Controlled

Trials

� Cohort Studies and Case Control Studies

� Case Reports and Case Series, Non-systematic observations

BIAS

Expert Opinion

Exp

ert O

pin

ion

Expert Opinion

BEST EVIDENCE

25.11.2016

2

The best evidence

Eminence based medicineEloquence based medicineEmotionbased medicineEvidence Based Medicine (old school)Evidence Based Medicine

1. addresses health outcomes and not just intermediate outcomes

2. is from “real” patients3. considers harms and

benefits4. fits the circumstances5. comes from well-designed,

well conducted studies

Some limitations of evidence based medicine

Many questions do not have answers!Evidence from populations - ?Relevance to individualTrials - not ‘real’ usageLack of local ownership of recommendationsClinical effectiveness vs cost effectiveness

Patients with multiple chronic conditions

• Those to whom most guidelines could apply, yet no guidelines ‘made’ for

them

• To enable stratified guidelines

– Need to generate science

– Ensure reporting and packaging off information

– Perhaps prioritize clusters off conditions

– Encourage guideline developers to take steps in this direction

Clinical Practice Guidelines (CPGs) and People with Multimorbidity

“Treating an Illness Is One Thing. What About a

Patient With Many?”

New York Times, March 31, 2009

Image: Brendan Smialowski for the New York Times

Prevalence of co-occurring chronic

conditions is high

CPGs developed for and emphasize single

disease perspective

Evidence-based Practice and Healthcare

Seven StepsStep 0 - Cultivate a spirit of inquiryStep 1 - Ask clinical question in PICOT format Step 2 - Search for the best evidenceStep 3 - Critically appraise the evidenceStep 4 – Integrate the evidence with clinical expertise and patient preferences and valuesStep 5 – Evaluate the outcomes of the practice decisions or changes based on evidenceStep 6 – Disseminate the result

Other models in use

Population,

Interest (area of )

Comparison intervention or group,

Outcome,

Time

Reassessment of clinical practice guidelines

Editorial by Shaneyfelt and Centor (JAMA 2009)-“Too many current guidelines have become marketing and opinion-based pieces…”“AHA CPG: 48% of recommendations are based on level C = expert opinion…”“…clinicians do not use CPG […] greater concern […] some CPG are turned into performance measures…”“Time has come for CPG development to again be centralized, e.g., AHQR…”

25.11.2016

3

Where GRADE fits in

Prioritize problems, establish panel

Systematic review

Searches, selection of studies, data collection and analysis

Assess the relative importance of outcomes

Prepare evidence profile:

Quality of evidence for each outcome and summary of findings

Assess overall quality of evidence

Decide direction and strength of recommendation

Draft guideline

Consult with stakeholders and / or external peer reviewer

Disseminate guideline

Implement the guideline and evaluateG

RA

DE Systematic review

Guideline development

P

I

C

O

Outcome

Outcome

Outcome

Outcome

Critical

Important

Critical

Not

Summary of findings

& estimate of effect for each outcome

Rate

overall quality of evidence across outcomes based on

lowest quality

of critical outcomes

RCT start high,

obs. data start low

1. Risk of bias

2. Inconsistency3. Indirectness

4. Imprecision

5. Publication bias

Gra

de

d

ow

nG

rad

e

up 1. Large effect

2. Dose response

3. Confounders

Very low

Low

Moderate

High

Formulate recommendations:

• For or against (direction)

• Strong or weak (strength)

By considering:

� Quality of evidence

� Balance benefits/harms

� Values and preferences

Revise if necessary by considering:

� Resource use (cost)

• “We recommend using…”

• “We suggest using…”• “We recommend against using…”

• “We suggest against using…”

Practical Clinical Trials

1. Compare clinically relevant interventions2. Enroll a diverse study population3. Recruit from a variety of practice settings4. Measure a broad range of relevant health

outcomes

PCTs vs. ECTs

Practical Clinical Trials Explanatory Clinical Trials

Hypothesis and study design are formulated based on information needed to make a decision

Designed to better understand how and why an intervention works

Addresses risks, benefits, and costs of an intervention as they would occur in routine clinical practice

Maximize the chance that biological effect of a new treatment will be revealed by the study

Models of Research

Traditional Model Patient‐Centered

Model (PCORI)

Audience Other researchers, maybe

policy makers.

End-users (patients, health care

providers,professional

organizations, policy makers)

Products Presentations and

publications in scholarly

journals

New tools, new evidence*

Patient

Involvement

Often none Essential in determining

acceptability to end-users

Stakeholder

Involvement

Often none Essential for partnering on

dissemination (eg, co- branding)

Mechanisms that allow patients to collect outcomes from home before or after an office visit have proved

helpful.

Improving patient engagement continues to be seen as a necessary step for improving the efficiency and

safety of care……. Health Affairs 35, no.4 (2016):575-582

*Reported in a manner understandable to each target audience.

Clinical benefit

a) Survival extensionb) Improvement in functionc) Quality of LifeCost Effectiveness – Never Considered

Post 1960� Evidence of Effectiveness

1. Defined as benefit to patients, not to doctors or

to society at large

� Safety to the patient

Efficacy Standards ‐ IOM

25.11.2016

4

1. Patient reported outcomes are UNEQUIVOCALPatient reported outcomes are difficult to blind, serial assessments are required

2. Soft end pointsObjective response ratesTime to progressionProgression free survivalDisease free survival

3. Clinical significance to changes of quality of life is unclear4. Combined tumor effects and quality of life are more tangible and

credible5. Using predictive biomarkers – for safety, response and

metabolism

What constitutes reasonable evidence of efficacy and effectiveness in cancer?

1. Evidence – that which tends to prove or disprove something, Ground of belief or proof

2. Criteria for evidence in absolute certainty, is difficult3. P-value is statistical4. Clinical trials are not perfect5. Evidence is often unavailable, inconclusive or contradictory

Efficacy – the capacity for producing a desired result or effect

What is the desired result?Full clinical benefit or a surrogate� Can it be measured precisely and reliably� Is the result transferred to community or real life

Effectiveness – how well a treatment works in practice

1. Most often oncology trials are not evaluated in community

2. Clear standards are not available

3. Collecting and reporting clinical effectiveness data are disorganized

4. Off label use is common

Randomized clinical trials are considered as Gold Standards

Not everything can be randomized

Other forms of analysis such as propensity score, statistical modelling may facilitate casual inference

Co-variates do not determine whether the patient do well or bad

Clinical components of Randomized trials

1. Designation of pre-specified hypothesis with primary, and secondary end points

2. Pre specified data cut-off for any continuous measurement to define what constitutes a positive or negative finding

3. Define sample sets with eligibility criteria that are as inconclusive as possible.

1. Power calculation to show that there is a reasonable probability of definitely answering the research question.

2. Un-bias – ascertainment of end points including blinding wherever possible and ethical protocol specified criteria and independent review of end points

3. Complete information through a standard follow-up schedule and a few patients lost to follow-up

25.11.2016

5

1. After being efficacious in RCTs, treatment are often refined to be studied in community where control environment is lost

2. Doses and schedules are changed3. Combination are made4. Drugs are often used off label

1. Refinements are rarely made2. Comparative effective research3. Meta-analysis on large scale4. Observational studies may prove some

validation

Tools are limited, hence no level 1 evidence

HOW TO BRIDGE ?

TWO SOLUTIONS

a) Cluster Randomizationb) Large simple pragmatic trials - QASARc) Evidence can be evaluated on a hierarchy of end

point strength – eg Overall Survivald) Use of biomarkers

−Predictive−Prognostic

Evidence based medicine (1)

Potential questions

• Is the clinical trial population representative of real-life?

• Cost-effectiveness over established (generic) comparator

• What do we know about the risk of polymedication?

• Will frail patients have the same benefit/risk profile?

• Costs and benefits for individual and society?

Challenges

• Balancing risk of involving frail/older patient in clinical trials

• Creating an “orphan” older population? (not authorising/not reimbursing)

• Design appropriate pharmacovigilance for unknowns

• Communicating to reduce inappropriate prescription

eCTD Module Age 65‐74number / total number (all ages)

Age 75‐84number / total number (all ages)

Age 85+number / total number (all ages)

Efficacy and Safety Studies

Human PK Studies

Human PD Studies

Biopharmaceutical Studies

MedDRA TermsAge <65

number (percentage)

Age 65‐74

number (percentage)

Age 75‐84number

(percentage)

Age 85+

number (percentage)

Total ADRs

Serious ADRs – Total

- Fatal

- Hospitalization/prolong existing hospitalization

- Life-threatening

- Disability/incapacity

- Other (medically significant)

AE leading to drop-out

Psychiatric disorders

Nervous system disorders

Accidents and injuries

Cardiac disorders

Vascular disorders

Cerebrovascular disorders

Infections and infestations

Quality of life decreased

Sum of postural hypotension, falls, black outs, syncope,

dizziness, ataxia, fractures

Evidence based medicine (2)

Assessment report‐ geriatric tables

Take home messages:

1) Qualitative not statistical-

focus attention of reviewer on

available data in relation to

epidemiology of disease

2) Adaptations might be

appropriate depending on

product/disease

3)statements made after

consideration of these data

should be meaningfully

reflected in the product

information.

25.11.2016

6

CER Concept and Definitions

Comparative Effectiveness Research Other definitions offer considerable overlap

Organization Definition

American College of PhysiciansEvaluation of the relative (clinical) effectiveness, safety and cost of 2 or more medical services, drugs, devices,

therapies, or procedures used to treat the same condition.3

Institute of Medicine (IOM) – Roundtable

on Evidence-Based Medicine

Comparison of one diagnostic or treatment option to ≥1 others. Primary CER involves the direct generation of

clinical info on the relative merits or outcomes of one intervention in comparison to ≥1 others. Secondary CER

involves the synthesis of primary studies to allow conclusions to be drawn.4

Agency for Healthcare Research and

Quality (AHRQ)

A type of health care research that compares results of one approach for managing a disease to results of other

approaches. CER usually compares ≥2 types of treatment, such as different drugs, for the same disease but it can

also compare medical procedures and tests. The results can be summarized in a systematic review.5

Medicare Payment Advisory Commission

(MedPAC)

Evaluation of the relative value of drugs, devices, diagnostic and surgical procedures, diagnostic tests, and medical

services. By value, it is meant the clinical effectiveness of a service compared with its alternatives.6

Congressional Budget Office (CBO)A rigorous evaluation of the impact of different options that are available for treating a given medical condition

for a particular set of patients. Such research may compare similar treatments, such as competing drugs, or it

may analyze very different approaches, such as surgery and drug therapy.7

Center for Medical Technology Policy

(CMTP)

The direct comparison of existing health care interventions to determine which work best for which patients and

which pose the greatest benefits and harms. The core question of comparative effectiveness research is which

treatment works best, for whom, and under what circumstances.8

CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to

prevent, diagnose,treat and monitor a clinical condition or to improve the delivery of care.The purpose of CER is to

assist consumers, clinicians, purchasers and policy makers to make informed decisions that will improve health care

at both the individual and population levels. Primary CER involves the direct generation of clinical info on the

relative merits or outcomes of one intervention in comparison to ≥1 others. Secondary CER involves the synthesis

of primary studies to allow conclusions to be drawn.4

Weighing the strengths and limitations of each type of comparative effectiveness research (CER) study.

Comparing Pragmatic vs. RCT and Observational

Characteristic RCT Pragmatic Observational

FocusEfficacy and safety; assess

mechanistic effect; Can it work

Effectiveness and safety; assess /

inform decision-making; Does it work

under usual care conditions?

Effectiveness and safety; Does it

work in actual practice?

Setting Ideal / artificialReal-world routine care (with

potential minor departures)Real-world routine care

Population Strictly defined; homogenous Typically broad; heterogeneous Broad; heterogeneous

Randomization Yes Typically yes No

Blinding Typically yes No No

Interventions Fully interventional Minimally interventional (e.g., rand.) Non-interventional

Outcomes Clinical surrogates; short term Longer term outcomes; PROs Long term outcomes; PROs

Sample Size Typically small Typically larger Typically large

ValidityHigh internal (↓ bias); low

external (↓ generalizability)

Moderate internal; moderate to high

externalLow internal; high external

Prospective/Retro Prospective Prospective Prospective or retrospective

Comparable cost Higher Moderate Lower

Example sub design Adaptive design LST; adaptive designDatabase studies, cohort, case-

control, cross-sectional

MCDA FOR HEALTHCARE

Multicriteria decision analysis (MCDA) is an application of analytical methods to help decision-makers to explicitly

consider(qualitatively or/and quantitatively)

multiple criteria to support their holistic integration to achieve a predetermined goal

35

MCDA – OPERATIONALIZING THE GOAL OF HEALTHCARE

EVIDEM Collaboration, a not-for-profit organization developing collaboratively an open source multipurpose MCDA-based approach translated in 10 languages and used throughout the world

www.evidem.orgg

Decision Tree

Criteria

Decision evidence MatrixProcess

“Doing

what is

best”

RootsEthical positions

Goal

25.11.2016

7

� Maximize efficacy/effectiveness

� Maximize safety

� Maximize patient perceived health/reported outcomes

� Best therapeutic benefit (e.g, cure)

� Best preventive benefit - Public health (eradication )

Imperative to help (deontology)

MCDA –TRANSFORMING ETHICAL POSITIONS INTO CRITERIA *

� Benefit greatest number (size of population)

� Economic consequences: savings on cost of intervention

� Economic consequences: savings on other medical cost

� Economic consequences: savings on non-medical cost

� Management of opportunity cost and affordability

� Alignment with mandate/scope of healthcare system

� Minimize environmental impact

Prioritizing those who are worst off (fairness, distributive justice)

� Alleviate suffering in patients with severe diseases

� Alleviate suffering in patients with no options (Unmet needs)

� Promote intervention for established priorities (e.g., vulnerable

populations, rare disease)

Greatest good for greatest number

(utilitarianism)

� Highly valid and relevant evidence

� Expert consensus

� Wise use of resources (see economic criteria above)

� Awareness of system capacity and appropriate use of intervention

� Awareness of common goal and specific interests

� Awareness of political, historical and cultural context

Practical wisdom & goodness (Virtue ethics)

*Each objectives can be justified by several ethical positions

“Doing what

is best”

Outcomes of intervention

Need for intervention

Type of benefit

Economic consequences of intervention

Knowledge on intervention

� Comparative efficacy/effectiveness

� Comparative safety

� Comparative patient reported outcomes

� Type of therapeutic benefit (e.g, cure)

� Type of preventive benefit - Public health

(eradication )

MCDA – STRUCTURING CRITERIA FOR OPERATIONALIZATION

“Doing what

is best”

Qualitative criteria (contextual)

� Disease severity

� Size of population (greatest number)

� Unmet needs

Quantifiable criteria (normative universal)

� Mandate/scope of healthcare system

� Established priorities (e.g., rare disease)

� Common goal and specific interests

� Environmental impact

� Opportunity cost and affordability

� System capacity and appropriate use of intervention

� Political, historical & cultural context

� Economic consequences: Cost of intervention

� Economic consequences: other medical cost

� Economic consequences: non-medical cost

� Relevance and validity of study data

� Expert consensus –clinical practice guidelines

Normative contextual

Feasibility

Procedural values• Participatory• Reflective• Transparent• Systematic

Highly synthesized evidence

39

MCDA – INTEGRATING EVIDENCE AND VALUES INTO A PROCESS

DECISION CRITERIA

Quantitative criteria Relative Weights

Disease severity � Low � High

Efficacy/effectivenessEtc

� Low � High

3 -WEIGTHS

Highly synthesized evidence

Turner syndrome: Female specific generic disorder characterized

by reduced life expectancy, cardiovascular defects, increased risk of diabetes, absence of puberty, infertility, defects in visuo-spatial

organization and non-verbal problem solving, and short stature (details)

APPRAISAL

Score

� High

�

�

� Low

4- EVIDENCE5 - PERFORMANCE

SCORES

Qualitative criteria

System capacity

etc

Impact

� negative� neutral� positive

6 - QUALITATIVE IMPACTS

• Collaborative• Adaptable• Holistic • Accountable

2 - CRITERIA

“Doing what

is best”

1 - GOAL

EVIDEM Collaboration, https://www.evidem.org/docs/2015/EVIDEM-v3-0-Decision-criteria-conceptual-background-definitions-and-

instructions-June-2015b.pdf

40

MCDA EVIDENCE MATRIX

INTERVENTION OUTCOMES CRITERIA HIGHLY SYNTHESIZED INFORMATION SCORES

Comparative efficacy / effectiveness

4 placebo controlled RCTs (2-year (toddlers) to 11-year treatments; N=42 to 104,

1 in Canada, 3 in USA): Final height of treated patients = 147 cm to 150 cm; difference with untreated = 7 cm

Observational controlled studies (2-year to 8-year treatments, N=26 to 123, 1 in Germany, 1 in Greece, 1 in Israel, 3 in Italy): Final height of treated patients = 148 cm to 151 cm; difference with controls = 2.1 to 6.8 cm (see details)

5 Major improvement 4 3 2 1 0 No improvement

-1 -2 -3 -4 -5 Lower than

comparators

Comparative safety

Common AEs (from RCTs –frequency at least twice of placebo): Surgeries (50%), ear problems (6 % to 47 %), joint (13.5%) and respiratory (11%) disorders, sinusitis (18.9%)

Serious AEs (from registries, no control data): Intracranial hypertension (0.2%), slipped capital femoral epiphysis (0.2 – 03.%), scoliosis (0.7%), pancreatitis (0.1%), diabetes mellitus (0.2 to 0.3%), cardiac/aortic events (0.3%), malignancies (0.2%)

Warnings: Scoliosis, slipped capital femoral epiphysis, intracranial hypertension, ear disorders, cardiovascular disorders, autoimmune thyroid disease, insulin resistance


-1 -2 -3 -4 -5 Lower than

comparators

Comparative patient-perceived health / patient-reported outcomes

Inconclusive data:

1 RCT (2-year treatment data, N=28, Canada): higher rating on questionnaire by GH treated patients versus untreated for some domains but not for others

2 observational studies : no significant differences on SF-36 dimensions in one study (5-year treatment, N=568, France) and significant differences in another (7-year treatment N=29, Holland); other questionnaires, non significant differences Convenience: Subcutaneous injection 3 days a week or daily


-1 -2 -3 -4 -5 Lower than

comparators

4- EVIDENCE5 - PERFORMANCE SCORES

INTERPRETATIONOF EVIDENCE BY EVALUATORS

Sir Rawlins,

NICE, HTAi

Seoul 2013:

“Accept that

interpretation of

data that takes

place during

appraisals

requires

judgement”

The Trillion dollar questions

1.How to collect evidence on value and then incorporate this evidence into decisions on coverage, reimbursement, and payment for healthcare services? 2.How to develop value-based, cost effective healthcare that is trusted and not perceived as only cost cutting for profit/balancing federal budget. 3.Lack of evidence is a real impediment to value-based healthcare. Collecting this data will take time. Policy decisions based on incomplete data is subject to serious negative consequences. Half truth sometimes more dangerous than nothing.

Comparative clinical effectiveness

Incremental cost for better clinical outcomes(long-term)

Other benefits ordisadvantages

Contextual considerations

“Care Value”

Public discussionand vote

HIGH INTERMEDIATE

LOW

“Care Value”

Public discussion and vote

Potential health system budget impact

(short-term)

Provisional“Health System Value”

Public discussion

NO VOTE OR FORMAL DESIGNATION

Maximizing Health System Value

Policy Roundtable discussion

HIGH INTERMEDIATE

LOW

25.11.2016

8

• Comparative clinical effectiveness reflects a joint judgment of the magnitude of the comparative net health benefit and the level of certainty in the evidence on net health benefit.

• Patient groups inform what outcomes are important, differencesacross severity, time in disease course, etc.

• Patient groups inform re: opportunities for using or generating real-world evidence

Comparative Clinical

Effectiveness

Incremental cost per

outcomes achieved

Other Benefits or

Disadvantages

Contextual

ConsiderationsCare Value

The cost-effectiveness plane

New treatment more effective but more costly

New treatment more effective

New treatment more costly

Not costly &very effective

Costly & not very effective

Maximum acceptable cost-effectiveness ratio?

• Long-term perspective on clinical outcomes and cost

• Costs from health system (payer) perspective – all health care costs

• Standard measures of health gain– Additional life-years gained– Improvement in quality of life– Cost per quality-adjusted life year gained, aka “cost per

QALY”


Effectiveness

Incremental Cost per

Outcomes Achieved

Other Benefits or

Disadvantages

Contextual


• Societal “willingness to pay”– WHO 1-3x per capita GDP ($50,000-$150,000)

• Individual “willingness to pay”– ~2 times annual salary ($100,000)

• “Opportunity cost” for the health system– ~1x GDP in UK, Latin America– Extrapolated ~$50,000 per QALY in the US

• ICER: $100,000-$150,000 per QALY


Effectiveness


Outcomes Achieved

Other Benefits or

Disadvantages

Contextual


• Patient groups and others asked about benefits or disadvantages offered by the intervention to the individual patient, caregivers, the delivery system, other patients, or the public that would not have been considered as part of the evidence on comparative clinical effectiveness.

– Methods of administration that improve or diminish patient acceptability and adherence

– A public health benefit, e.g. reducing new infections– More rapid return to work or other positive effects on productivity (if not

considered a benefit as part of comparative clinical effectiveness)

• To be judged not by ICER but by one of its independent public appraisal committees


Effectiveness


Outcomes Achieved

Other Benefits or

Disadvantages

Contextual


• Contextual considerations include ethical, legal, or other issues that influence the relative priority of illnesses and interventions.

• Specific issue to be asked of patient groups and others:– Is this a condition of notably high severity for which other

acceptable treatments do not exist?– Are other, equally or potentially more effective treatments nearing

introduction into practice?– Would other societal values accord substantially more or less

priority to providing access to this treatment for this patient population?

• To be judged not by ICER but by one of its independent public appraisal committees.


Effectiveness


Outcomes Achieved

Other Benefits or

Disadvantages

Contextual


25.11.2016

9





“Care Value”


HIGH INTERMEDIATE

LOW

“Care Value”



(short-term)


Public discussion



HIGH INTERMEDIATE

LOW

Copyright ICER 2016





“Care Value”


HIGH INTERMEDIATE

LOW

“Care Value”



(short-term)


Public discussion



HIGH INTERMEDIATE

LOW

Copyright ICER 2016

There are Multiple ValueFrameworks

Clinical benefit: OS>PFS>RR

Toxicity: add points if less toxic, subtract if more

Net Health Benefit

Net Health Benefit and Cost: the ASCO Framework Two versions: advanced and curative contexts

Comparison in a trial: test vs standard

Cost (to system and to the patient)

Bonus: Extended survival

Calculating Clinical Benefit (modification)

Step 1: Determine the regimen’s CLINICAL BENEFIT

1.A. Is Hazard Ratio (HR) for deathreported?

YES. Assign an HR Score for death by subtracting the HR from 1, and thenmultiplying the result by 100. Write this number in the box labeled, “HR Score

(death)”. Proceed to 1.F.HR Score (death)

No. Proceed to 1B.

1.B. If HR for death is not reported, ismedian Overall Survival(OS)

reported?

YES. Assign an OS Score by calculating the % difference in median overall survival between the two regimens. Write this number in the box labeled, “OS Score”.

Proceed to 1.F.OS Score

NO. Proceed to 1.C.

1.C. If median OS is not reported, isHazard Ratio (HR) for disease

progression reported?

YES. Assign an HR Score for disease progression by subtracting the HR from 1, multiplying the result by 100, and then multiplying this number by 0.8. Write this

number in the box labeled, “HR Score (progression)”. Proceed to 1.F.

HR Score (progression)

NO. Proceed to 1.D.

1.D. If HR for disease progression is not reported, is median Progression-

Free Survival(PFS) reported?

YES. Assign an PFS Score by calculating the % difference in median progressionsurvival between the two regimens. Multiply this number by 0.8. Write this number

in the box labeled, “PFS Score”. Proceed to 1.F.

PFS Score

NO. Proceed to 1.E.

1.E. If median PFS is not reported, isResponse Rate (RR) reported?

YES. Assign an RR Score by adding the complete response (CR) and partial response (PR) rates, and then multiplying this number by 0.7. Write this number in

the box labeled, “RR Score.” Proceed to 1.F.

RR Score

1.F. Calculate the Clinical BenefitScore

Insert the HR, OS, PFS, or RR Score. Note: You should have a score for only 1 ofthe clinical benefit scales above. Write the total in the box labeled “Clinical Benefit

Score”. The maximum allowable points are 80 . Proceed to Step 2.Clinical Benefit Score

JCO, July 2016

25.11.2016

10

Calculating Toxicity Score(Modification) Grade and frequency matter

Step 2: Determine the regimen’s TOXICITY

Does the new regimen represent animprovement in toxicity over the

standard of care/ comparator?

For each of the regimens being assessed, compare the number and frequency of clinically relevant toxicities, and assign a Toxicity Score (-20 through +20) as shown below. Each clinically meaningful toxicity (i.e.,

exclude laboratory results only) is assigned a score between 0.5-2.0 based on grade and frequency.· For every Grade 1 or 2 toxicity with a frequency <10%, add 0.5 points.

· For every Grade 1 or 2 toxicity with a frequency greater than or equal to 10%, add 1.0 points.· For every Grade 3 or 4 toxicity with a frequency <5%, add 1.5 points.

· For every Grade 3 or 4 toxicity with a frequency greater than or equal to 5%, add 2.0 points.

Calculate the total number of toxicity points for each regimen. Calculate the % difference in total toxicity points between the two regimens, then multiply by 20 to obtain a toxicity score. If the regimen being

evaluated is more toxic than the comparator, subtract the toxicity score of the regimen from the clinical benefit score. If the regimen is less toxic than the comparator, add the toxicity score of the regimen to the

clinical benefit score. If there are unresolved symptomatic treatment-related toxicities at 1 year after completion of treatment, subtract 5 additional points from the clinical benefit score.

Proceed to Step 3.

JCO, July 2016

Calculating Bonus Points modification

JCO, July 2016

Step 3: Determine Bonus Points

3.A. TAIL OF THE CURVE. At a timepoint that is double the median overall

survival (or median PFS if mOS is not reported) of the comparator regimen,

is the fraction of patients surviving on the comparator regimen at least 20%,

AND is there at least a 50% relative improvement in the fraction of patients

surviving?

YES. If yes, award 20 points, and place this number in the box labeled “Tail of the Curve Bonus Points”. Proceed to Step 3.B.

Tail of the Curve Bonus Pts

NO. No bonus points are awarded. Proceed to Step 3.B.

3.B. PALLIATION BONUS. Are data

related to the palliation of symptoms

reported?

YES. If a statistically significant improvement in cancer-related symptoms is

reported, award 10 points, and place this number in the box labeled “Palliation Bonus Points”. Proceed to Step 3.B.

Palliation Bonus Pts

NO. No bonus points are awarded. Proceed to Step 3.C.

3. C. TREATMENT-FREE INTERVALBONUS. Are data related to treatment-

free interval reported?

YES. If a statistically significant improvement in treatment-free interval is reported,

award points based on the table below, and place this in the box labeled “Clinical Benefit Bonus Points”. This is the interval from completion of study treatment to resumption of treatment. Proceed to 3.D.

Treatment-Free Interval Bonus

Bonus Points 0 5 10 15 20

% Change >0-19 20-35 36-49 50-74 >/=75

NO. No bonus points are awarded. Proceed to Step 3.D. Total Bonus Pts

3.D. Calculate Total Bonus Points Add the Palliation Bonus Points (Step 3.A) and the Treatment-Free Interval Bonus

Points (Step 3.B). Write this number in the box labeled “Total Bonus Pts”. The maximum points available for Bonus Points is 50. Proceed to Step 4.

Step 4: Determine the regimen’s NET HEALTH BENEFIT

Calculate the Net Health Benefit Add the Clinical Benefit Score (Step 1), Toxicity Score (Step 2), and Bonus Points

(Step 3). This yields a Net Health Benefit Score. Write this number in the box labeled “Net Health Benefit”. The maximum points available for Net Health Benefit are 150 (100 + 50 bonus points). Proceed to Step 5.

Net Health Benefit

Calculating Net Health Benefit Juxtaposed against Cost per Month

JCO, July 201635http://www.ncbi.nlm.nih.gov/pubmed/26028407

Nivolumab vs docetaxel

ClinicalBenefit Score

HR (death) = 0.59Clinical benefit = (1 – 0.59) x100 x1 = 41

41

Toxicity Score

Bonus Points

Palliation 0

Treatment-Free Interval 0

Total Bonus Points 20

Net Health Benefit 73.2

DAC $11,699

Cost/NHB 160

Nivolumab

5.5

34.5

0

13

Docetaxel

12.2

Toxicity score: 13/33.5 – 1 = -61%-0.61 * (-20) = 12.2

Tail of the Curve 20

7 x 0.5 = 3.59 x 1 = 9

10 x 1.5 = 153 x 2 = 6

Total 33.5

Gr 1-2 <10%Gr 1-2 >10%

Gr 3-4 <5%Gr 3-4 >5%

CheckMate 017: Nivolumab versus docetaxel

in advanced squamous NSCLC

11 x 0.5 =3 x 1 =

3 x 1.5 =0 x 2 =

Total

1

N Engl J Med 2015; 373:1627-1639

NCCN Evidence Blocks™

NCCN Evidence Blocks

© NationalComprehensiveCancerNetwork,Inc. 2016, All rights reserved.The NCCN Guidelines® andthis illustrationmay not be reproduced in

any formwithout the express written permissionof NCCN®.

• Use consistent methodology and display to inform decision-making

• Measures

– Efficacy

– Safety

– Quality of Evidence

– Consistency ofEvidence

– Affordability

• More shading is better

E S Q C A

25.11.2016

11

Efficacy of Regimens Scale



Score Summary Definition

5 Highly effective

Often provides long-term survival advantage or curative potential

4 Very effective

Sometimes provides long-term survival advantage or curative potential

3 Moderatelyeffective

Modest, no, or unknown impact on survival but oftenprovides control of disease

2 Minimally effective

Modest, no, or unknown impact on survival and sometimes provides control of disease

1 Palliativeonly

Symptomatic benefit only

Affordability of Regimens Scale



Affordability refers to overall cost of an

intervention including drug cost, required

supportive care, infusions, toxicity monitoring,

management of toxicity, probability of care being

delivered in the hospital

Score Summary/Definition

5 Very inexpensive

4 Inexpensive

3 Moderately expensive

2 Expensive

1 Very expensive

ESMO ValueFrameworks

Factors taken into account for ESMO-MCBS

Magnitude of Clinically Benefit

Overall survival,

Progression free survival

Toxicity

Costs

Prognosis of the

condition

Quality of Life

HR,

Long term survival,

RR

Cherny, N et al, Ann Oncol epub 30 May 2015

25.11.2016

12

Considering the Several Frameworks?

• Shall we strive for a uniform approach to determining clinicalbenefit”?

• Cross- trial comparisons are necessary if clinical benefit assessments are to have meaning

• Reasonable cost; after accounting for net health benefit and cost

-threshold for value – how much will we/can we spend forwhat degree of gain?

- is there a role for value-based pricing?

• How to incorporate the value of agents/regimens into clinicalpathways?

• Shared decision making

The Goal

• All patients with cancer will have lifelong access tohigh-quality, effective, affordable andcompassionate care

• The most accurate cancer information will beavailable so that patients and physicians can makeinformed decisions about cancer prevention andtreatment (shared decision-making)

• The most accurate cancer information will beavailable so that policy makers will make informeddecisions based upon the value to be delivered by cancer prevention and treatment

Discussion Good News….

Our future’s so

bright we gotta

wear shades

Evidence as the Basisfor Clinical Policy

Clinical

GuidelinesEvidence

FormularyClinical

Processes & Systems

Appropriateness

Measures

Decision

SupportPerformance

Measures

Clinical

Reminders

Documents

Pre0174-Bhattacharyya Gouri Shankar