© CDISC 2012
1
Institute of Medicine Workshop:
DIGITAL DATA PRIORITIES FOR
CONTINUOUS LEARNING
IN HEALTH AND HEALTH CARE
“Data Quality in Clinical Research”
23 March 2012
Rebecca D. Kush, PhD
President and CEO, CDISC
© CDISC 2012 2
© CDISC 2012
JPMA Presentation –
Data Quality for Clinical Research
3
© CDISC 2012
Qu
ality
Re
qu
ire
me
nts
Type of Project
Non-medical
Sales and
Marketing
Signal or
Trend
Detection
Regulated
Research
Basic
Research
Signal
Validation;
Active Safety
Surveillance
“Sushi-grade”
“All you can eat
buffet grade”
Clinical
Decisions
SP
EC
UL
AT
ION
by
R.D
. K
us
h
© CDISC 2012
Key Messages
• Regulated Clinical Research requires ‘sushi-
grade’ data quality
• Building quality in from the beginning is ideal
• Data standards improve data quality
5
© CDISC 2012
Healthcare and Clinical Research:
Parallel Universes
Source: Landen Bain, CIO, DUMC ~ 2004
© CDISC 2012 7
“The Plight of the Site”
• For clinicians doing research today…
~ 50-60 % of trials - data collected on paper (3- or 4-
part NCR paper)
~ 40-50 % of data are collected by eClinical “point
solutions” (electronic data capture tools)
data are entered 4-7 times total, 2-3 times by the
clinicians or study coordinators
An average active study site has 3 disparate solutions
• To report an unexpected or serious adverse
event does not fit into normal clinical care
workflow and takes excessive time.
• Most clinicians do one regulated clinical research
study and no more.
8
Current Typical Research Data Flow
© CDISC 2012 9
Data Quality in Clinical Research: Cost of Error Correction
Site
~$30
Co
st
of
Qu
ality
$
(1
99
5)
Point in Process for Error
Detection/Correction (per error)
CRO
~$75
Stats-
Tables
Listings
>$5,000
QA of
Final
Report
>$8,000
Database
Lock
© CDISC 2012
CFR - Code of Federal Regulations
Title 21 - FDA
10
TITLE 21--FOOD AND DRUGS
CHAPTER I--FOOD AND DRUG ADMINISTRATION
DEPARTMENT OF HEALTH AND HUMAN SERVICES
SUBCHAPTER A--GENERAL
PART 11 ELECTRONIC RECORDS; ELECTRONIC SIGNATURES
The regulations in this part set forth the criteria under which
the agency considers electronic records, electronic signatures,
and handwritten signatures executed to electronic records to be
trustworthy, reliable, and generally equivalent to paper records
and handwritten signatures executed on paper.
© CDISC 2012 11
Mapping (Source to Submission)
© CDISC 2012 12
… but also Submission to Source
Trustworthiness
© CDISC 2012 13
ODM & Audit Trail
What
Why
Who
When
© CDISC 2012 14
The “Road” to Quality Clinical Data
• Build quality into the system
• Train and educate
site personnel, project team and reviewers/auditors
• Decrease the amount of data collected
• Define the data set needed and specify requirements
• Standardize formats and procedures
• Also plan for data quality during post-marketing
• Decrease the number of times data are ‘handled’
(Note: Anticipated ‘by-products’ of these steps to improve
quality are increased efficiency and lower costs.)
Source: Assuring Data Quality and Validity in Clinical Trials
for Regulatory Decision Making: Workshop Report, 2000
© CDISC 2012 15
Optimizing the Process
auto
reconciliation
data
conception
(e)CRFs (e)Source
Documents
EHR
eSource Healthcare
Delivery Clinical
Research
~ 1997
B. Kisler, R. Kush
© CDISC 2012
CRO or Partner
Reviewers (e.g. Research Partner,
Sponsor, Registry, Regulator, IRB, DSMB,
Quality Measures)
Public Registries,
IRB, DSMBs EDC
EHR
Data Sources
Care and/or Research Site (Healthcare Location,
Investigator, Site Personnel)
Study Sponsor (e.g. ARO, CRO, Vendor, Principal Investigator, potentially AHRQ…)
Std. Common
Research
Dataset (+)
EHR
De
-id
en
tifi
ed
Dat
a Site
Research
Archive
Research Results,
eSubmission
Standard Formats
Patient Value: Quality of Healthcare, Safety
Research informs healthcare more effectively Build quality into process at beginning
Regulatory Authority
Scientific
Pub-
lication
Continuity
of Care
Doc RFD*
Interoperability
Specification
© CDISC 2012
Press Release: June 10, 2010
Copyright C-Path 2011 17
Initial database required mapping to a
standard (CDISC); can now be leveraged
to collect data using the standard
Database now has >6,000 research
study patients
Now being used by > 200 qualified
research teams in 35 countries
Efficient collaboration
Better science
Regulatory efficiency
Why Standards ?
18
ATM
Clinical Trials
Date of Birth
Jan. 15, 2011
January 15, 2011
1/15/11
1/15/2011
15/1/11
15 January 2011
15-1-11
2011-1-11
Clinical Trials
What is the average
age of patients
enrolled in 11 industry
trials for Alzheimer’s
Disease?
© CDISC 2012
What was learned?
ADAS-Cog Variability
ADNI J&J Wyeth sanofi-aventis Pfizer AstraZeneca Abbott
Item 1 Word Recall Word Recall Word Recall Word Recall Word Recall Word Recall Word Recall
Item 2 Commands Name Obj/fing. Name Obj/fing. Commands Name Obj/fing. Name Obj/fing. Name Obj/fing.
Item 3 Constr. Praxis Delayed recall Commands Constr. Praxis Commands Commands Commands
Item 4 Delayed recall Commands Constr. Praxis Delayed recall Delayed recall Constr. Praxis Constr. Praxis
Item 5
Naming Obj/fing. Constr. Praxis Idea Praxis Name Obj/fing. Constr. Praxis Idea. Praxis Idea. Praxis
Item 6 Idea. Praxis Idea Praxis Orientation Idea. Praxis Idea. Praxis Orientation Orientation
Item 7 Orientation Orientation Word Recog Orientation Orientation Word Recog Word Recog
Item 8 Word Recog. Word Recog. Remem. Instr. Word Recog Word Recog Remem. Instr. Spoken Lang Abil.
Item 9 Remem Instr. Remem Instr. Spoken Lang. Abil. Remem. Instr. Remem. Instr.
Spoken Lang. Abil. Comprehension
Item 10 Comprehension Spoken Lang. Abil.
Word Finding Dif.
Spoken Lang Abil.
Spoken Lang Abil.
Word Finding Dif.
Word Finding Dif.
Item 11
Word Finding Dif.
Word Finding Dif. Comprehension
Diff. Spont. Speech
Word Finding Dif. Comprehension Remem. Instr.
Item 12
Spoken Lang. Abil. Comprehension Concentration Comprehension Comprehension Concentration
Item 13 Number cancel. Concentration Concentration Concentration
© CDISC 2012
Now published….
20
CDISC Alzheimer’s Disease SDTM User Guide (Version 1.0) ©
See also “Questionnaires” on the CDISC website
under Standards & Innovations.
CAMD Cognition Test Data
10-Year Disease Progression
by Severity at Entry
Mean (line) and 90% Credible Intervals* (gray shaded area)
* the posterior probability of the average individual's mean ADAS-cog being in the interval is 90%
Mild Moderate Severe
ADAS-cog
© CDISC 2012
• Data from individual clinical trials of 200-400 patients with Alzheimer’s
Disease had limited power and frequently failed due to variability in
outcome and small sub-groups.
• Mapping control arm data to a standard format for >6,000 patients in 20
trials created a dataset with higher quality (common methodology for
ADAS-cog) and greater power to assess variables affecting progression
Severity and age at entry
ApoE4 genotype
• Data standards can
1. Increase learning from clinical research study analysis
2. Facilitate data sharing across research studies
3. Create databases with which to design more informative and
efficient research studies
NOTE: Data standards are most valuable (significantly reducing time and
resources) when implemented from the beginning.
23
Conclusions from CAMD
© CDISC 2012 24
Value of Standards
Using the data wisely……
Making sure it is
accurate, that it is
used as agreed,
and that we can find it!
We owe it to the patients
who agree to participate
in research studies and
share their data.
© CDISC 2012
Key Messages
• Regulated Clinical Research requires ‘sushi-
grade’ quality data
….and this translates to a robust quality process that is
not easy to execute and must be trustworth
• Building quality in from the beginning is ideal
….adding in quality at the ‘back end’ (e.g. through
mapping or normalization) can certainly be done but
only at a high cost (time and resources)
• Data standards improve data quality
…..especially when implemented in the data collection
steps at the beginning
25
26
© CDISC 2012
Research findings to inform
healthcare decisions
Information from healthcare (private, aggregated)
to enable research
•Discovery of new therapies •Understanding diseases •Assessing efficacy •Monitoring safety •Public health/quality evaluations •Understanding responses (genomics, biomarkers) •Testing/comparing therapies (CER) •Post-marketing surveillance
•Quality healthcare •Informed decisions •Personalized medicine •Patient safety and privacy •Public health •Improved therapies •Efficiencies/reduced costs
Research Healthcare
© CDISC 2012 28