Upload
samuel-cobb
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Ida Sim, MD, PhD
March 4, 2008
Division of General Internal Medicine, and Center for Clinical and Translational Informatics
UCSF
Informatics for Clinical Research
Copyright Ida Sim, 2008. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.
2
Draft
for N
RC com
mitt
ee re
port,
do
not c
ite o
r circ
ulat
e Big Picture of Health Informatics
Virtual Patient
Transactions
Raw data
Medical knowledge
Clinical research
transactions
Raw research
data
Dec
isio
n su
ppor
t
Med
ical
logi
c
PATIENT CARE / WELLNES RESEARCH
Workflow modeling and support, usability, cognitive support, computer-supported cooperative work (CSCW), etc.
CTMSs
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Outline
• Clinical Trial Management Systems (CTMS)
– NCI/NIH vision
– case study: UCSF Cancer Center
• Naming data• Running trials on the Web• Summary
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Biomedical Informatics Spheres
Administrative Clinical Care Research
ClinicalBilling
Physical Networking
Standard Communications Protocols (e.g., HL-7)
Standard Vocabulary
PracticeManagement
Systems
Medical BusinessData Model
ElectronicMedicalRecord
Clinical CareData Model
??
??
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
CTMS
Administrative Clinical Care Research
ClinicalBilling
Physical Networking
Standard Communications Protocols (e.g., HL-7)
Standard Vocabulary
PracticeManagement
Systems
ElectronicMedicalRecord
Clinical Trial Management
Systems
Medical BusinessData Model
Clinical CareData Model
Clinical StudyData Models
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
EHR vs. CTMS
• EHR• Patient demographics• Chart notes
– problem list
• Visit and assessment• Lab and other orders• Results reporting• Clinical decision-making• Discharge summary
• CTMS• Title, NCT #, IRB #• Protocol document
– interventions, design, outcomes, etc.
• Study assessment• Outcomes assessment• Case report forms• Data analysis• Trial reporting/publication
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Clinic 2008
FrontDesk
Radiology
MedicalInformationBureau
Walgreens
Pharm BenefitManager
Benefits Check(RxHub)
HealthNet
B&T
UCare
Specialist
ReferralAuthorization
Internet Intranet Phone/Paper/Fax
Lab
UniLab
(HL-7)
IRB
Trial Design
Protocol
Funding Agency
Site 1 Site 2 Site 3
Site Management Organization (SMO)
Study DB
Data analysis
Results reporting
Contract R
esearch Organ
ization
(CR
O)
SponsorsAcademic PIs
?
Clinical Research Today
• >80% on paper
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Market Facts
• In early 90s, 80% trials done in academia, now reversed
• Huge business– Big Pharma: $15.9 billion on clinical trials [2006 PhRMA Industry
Profile]
– NIH budget $28.6billion 2005
• Fragmented, global industry– estimated 1200 organizations involved in clinical research in 2004 in
US (Sponsors, CROs, SMOs, AHCs...)
– CROs >$8b market in 2002, enrolled 20 million subjects in 2001
– “top US firms projecting...within 2 to 3 years as much as 65% of their FDA-regulated clinical trials will be conducted abroad” [Tufts Outlook 2006]
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Clinic 2008
FrontDesk
Radiology
MedicalInformationBureau
Walgreens
Pharm BenefitManager
Benefits Check(RxHub)
HealthNet
B&T
UCare
Specialist
ReferralAuthorization
Internet Intranet Phone/Paper/Fax
Lab
UniLab
(HL-7)
IRB
Trial Design
Protocol
Funding Agency
Site 1 Site 2 Site 3
Site Management Organization (SMO)
Study DB
Data analysis
Results reporting
Contract R
esearch Organ
ization
(CR
O)
SponsorsAcademic PIs
?
Want “Interoperability”
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Interoperability
• Ability of two or more systems or components to exchange information and to use the informationthat has been exchanged [IEEE Standard Computer Dictionary, 1990]
– syntactic: grammar, composition of what is said• e.g., using an exchange protocol
• e.g., HL7, DICOM, XML Document Type Definition (DTD)
– semantic: meaning of what is said• e.g., using a controlled vocabulary aka dictionary
• e.g., SNOMED, ICD-9
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
2 Routes to Interoperability
• “Microsoft” approach– one end-to-end system from trial inception to end
– used by all PIs, industry, CROs, worldwide
• Modular, component-based, interfaces approach– define common terms, models, interchange
protocols
– provide software components for assembly
– provide way to “certify” compatibility of systems
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Clinical Trial Software Components
Clinical Trial Management Infrastructure
Adverse Event Reporting
Study Calendar
Routine Data Exchange Financial Billing
Structured Protocol
Representation
Laboratory Interfaces
Clinical Trials DatabaseSystems
Interoperation
CRF Standardization
InvestigatorAnd
Site Credentialing
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
NCI Priority for Interoperation
Clinical Trial Management Infrastructure
Adverse Event Reporting
Study Calendar
Routine Data Exchange Financial Billing
Structured Protocol
Representation
Laboratory Interfaces
Clinical Trials DatabaseSystems
Interoperation
CRF Standardization
InvestigatorAnd
Site Credentialing
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
NCI caBIG Vision
caBIGTM Architecture WorkspacecaBIGTM Architecture Workspace
caBIGTM Vocabularies and Common Data Elements WorkspacecaBIGTM Vocabularies and Common Data Elements Workspace
Strategic Planning
Workspace
Strategic Planning
Workspace
Training Workspace
Training Workspace
Integrative Cancer
Research
Workspace
Integrative Cancer
Research
Workspace
In Vivo Imaging
Workspace
In Vivo Imaging
Workspace
Tissue Banks &
Pathology Tools
Workspace
Tissue Banks &
Pathology Tools
Workspace
Data Sharing & Intellectual
Capital Workspace
Data Sharing & Intellectual
Capital Workspace
Clinical Trials
Management Systems
Workspace
Clinical Trials
Management Systems
Workspace
15
CTMS Workspace Goals
• Facilitate the planning and instantiation of clinical trials,
(and monitoring of trials once they are instantiated)
• Facilitate the conduct of clinical trials
• Facilitate the reporting and sharing of clinical trial data
to existing/ new destinations
• Achieve interoperability
• Increase the ability of systems to access and use the
data and functionality of other systems
• Facilitate the integration of new sources and destinations
of data
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Critical Components
• Terminologies/vocabularies– base terms used to describe biomedical concepts
• e.g., SNOMED, NCI Thesaurus
• Common Data Elements– clinicallly-agreed upon data items for research
• e.g., “menopause” defined a certain way
• Common data model of study protocol– study information: e.g., eligibility criteria, treatment, outcomes
• CTOM, SDTM, BRIDG, etc. etc.
• Common interchange standards– e.g., CDISC (“HL7 for clinical research”)– so design, monitoring, reporting systems, etc. can talk
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
NCI-->CTSA-->NIH xxBIG??
caBIGTM Architecture WorkspacecaBIGTM Architecture Workspace
caBIGTM Vocabularies and Common Data Elements WorkspacecaBIGTM Vocabularies and Common Data Elements Workspace
Strategic Planning
Workspace
Strategic Planning
Workspace
Training Workspace
Training Workspace
Integrative Cancer
Research
Workspace
Integrative Cancer
Research
Workspace
In Vivo Imaging
Workspace
In Vivo Imaging
Workspace
Tissue Banks &
Pathology Tools
Workspace
Tissue Banks &
Pathology Tools
Workspace
Data Sharing & Intellectual
Capital Workspace
Data Sharing & Intellectual
Capital Workspace
Clinical Trials
Management Systems
Workspace
Clinical Trials
Management Systems
Workspace
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Outline
• Clinical Trial Management Systems (CTMS)
– NCI/NIH vision
– case study: UCSF Cancer Center
• Naming data• Running trials on the Web• Summary
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Commercial CTMSs• Functions
– document management (protocol, CRFs (Case Report Forms))
– finances, IRB– study calendar (what to do to whom when)– data management and analysis– reporting
• Examples– proprietary
• OracleClinical, C3D, Velos, etc. etc.
– open source• OpenClinica
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Cancer Center Needs
• Reports need to continue as an NCI-designated Cancer Center– e.g., trials initiated, pt demographics
• UCSF CC needed good data for reports– bought Velos eResearch in 2003 for reporting
– required all CC investigators to use Velos• licenses issued to PIs in CC and some in GCRCs
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
What Gets into Velos?
• Study summary data, as free text– administrative (PI, contact info, etc)– protocol (treatments, outcomes, etc)– IRB-related
• submitted protocol, approval status, current protocol• tied into Cancer Center PRS review, but not CHR
• Patient data– PI can define CRFs, data entered would be stored
in a Velos database• basic report functions• export to SAS or Excel for analysis
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
How is it Working?• Adoption
– rocky initially, better since integrated with PRS • For NCI reporting, successful• For data hosting and analysis
– just started this in 2007(!)– “easy to get data in, hard to get data out”
• biostatisticians are not too happy
• underlying data schema and access are opaque, by design
• less reponsive/customizable to reporting and analysis needs
• Not used for finances, study calendar, standardization of terms, etc.
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Outline
• Clinical Trial Management Systems (CTMS)
– NCI/NIH vision
– case study: UCSF Cancer Center
• Naming data• Running trials on the web• Summary
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Data Elements• To ensure sharable data, research data
variables should be standardized– standardized in naming
• terms should be from a controlled vocabulary (e.g., SNOMED, NCI Thesaurus)
– standardized to be common across studies• e.g., menopause with full clinical definition, to be used in
all breast cancer trials
• Both can be standardized through common Case Report Forms (CRFs)
Use “Add” button to list multiple symptoms
SNOMED Concept has been added to formRichesson R, 2007
SNOMED-CT Browser for Research
Implementation Status• > 300 electronic case report forms (CRFs)
• > 26,000 questions
• ~ 400 SNOMED CT and RxNorm browser contexts– 75% represent current findings (e.g., context is physical exam,
clinical assessment forms) and 25% represent historical findings (e.g., medical history context).
• 200 investigators and research staff have been trained on this tool
– Compliance is high (up to 90%)
– Satisfaction seems high (few complaints)
Richesson R, 2007
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Standardization Across Studies
• Studies fundamentally done to demonstrate differences between interventions, by types of patients [Clarke M, Trials 2007]
– common outcome measures necessary for pooling / systematic reviews
• e.g., 5-year cancer free survival, common asthma measures– also common eligibility criteria, e.g., Post-menopause
• post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy and not currently receiving therapy with LH-RH analogs [eg. Zolades])
• post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy)
• pre (<6 mo since LMP AND no prior bilateral ovariectomy, AND not on estrogen replacement)
• above categories not applicable AND Age >=50
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
NCI Approach in Cancer
• NCI caDSR (Data Standards Repository)– library of Common Data Elements (CDEs) that
others have defined– you can define new CDEs using terms from NCI
Thesaurus
• Let’s go search...– http://cdebrowser.nci.nih.gov/CDEBrowser/
– http://cdebrowser.nci.nih.gov/CDEBrowser/common/help/cdeBrowserHelp.html
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Case Report Forms
• Why reinvent the wheel for common forms?• caDSR has a Forms Builder
– pull together CDEs into CRFs• can download as HTML, XML, Excel, Word, PDF
• NCI now building library of CRFs – e.g., Demographics CRF built from CDEs
• PDF, Word, etc. or directly to CTMS for direct data entry
• Velos– can design CRFs in Velos, with direct access to NCI’s CDEs– if caBIG compatible, could in future access NCI’s CRF
library directly
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Outline
• Clinical Trial Management Systems (CTMS)
– NCI/NIH vision
– case study: UCSF Cancer Center
• Naming data• Running trials on the web• Summary
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Internet vs. Web
itsa
medicine
ucsf.edu
nci.nih.gov cochrane.uk myhome.com
Main Trunk Cables
local trunk cablethrough Berkeley
amazon.com
at homedial-in to itsa.ucsf.edu via modem
pacbell.net
aol.com
Internet Service Provider (ISP)via DSLor cable
LAN
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Internet vs. Web
• Internet = network of networks– computers and cables all linked to one another
and talking to one another using protocols
– supports lots of different internet protocols• e.g., http, ftp, smtp, https, rdf, doi, etc. etc.
• Web is the internet traffic that uses http– servers send out information in HTML
• Hypertext Markup Language
– web browsers can decode HTML and display it
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Clients and Servers
itsa
medicine
ucsf.edu
nci.nih.gov cochrane.uk myhome.com
Main Trunk Cables
amazon.com
at home
pacbell.net
aol.com
LAN
Server
Client
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Research IT on Internet/Web
• Research IT using Internet– uses Internet network of networks to send data and
commands back and forth– servers and clients do the storage, query, retrieval,
computation, reporting– may have nothing to do with a web browser
• Research IT using Web– web servers send HTML content over the Internet using
HTTP– web browsers and other “clients” receive that content for
display or computation• What are logistical and methodological issues?
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Eg: Smoking Cessation Trial
• Target audience– English and Spanish-speaking smokers
• Pre- and post demographic, etc. survey• Randomized Interventions
– downloadable brochure vs. brochure + email reminders + diary
• Outcome– quit rate
Slides from Ricardo Muñoz, [email protected] World Health Research Center, www.health.ucsf.edu
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Surveys on the Web
• Survey design and deployment• Recruitment
– selection, sampling, non-response bias
• Sample size calculation
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Web Surveys are Cheaper
• Web surveys have higher fixed cost but cost per additional respondent is much lower– marginal cost per mail survey respondent $1.93– phone $40 to $100– web $0
• Buy or build?– buy: many companies offer survey design,
deployment, and data management services– build: do-it-yourself
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Buying Survey Services
• Many, many companies exist• Survey Monkey www.surveymonkey.com
– free for 10 questions, 100 responses per survey– professional subscription $19.95/mo or $200/yr
• up to 1000 responses per month, $0.05 per additional response
• DatStat’s Illume – web-based survey creation and management– real-time data access and complex query capabilities– exports data to SAS, SPSS, etc. – Internet World Health Research Center is beta user
• $7000/yr first year, $3000/yr thereafter
• $4000 license/user (e.g., you)
Disclosure: I have no ties to SurveyMonkey or DatStat
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Survey Design• Usual survey design issues apply, PLUS• Technical design of survey
– platform (e.g., Mac) and browser (e.g., Safari) incompatibilities
– use Flash, Java, etc requiring plug-ins or version compatibility
– readiblity (font too small), need to scroll, confusing navigation, bugs
• What technology does respondent group use?– check some browser statistics sources
• e.g., http://www.w3schools.com/browsers/browsers_stats.asp
– need to test and double-test in various platforms and browsers used, various versions of HTML, Java, Flash, etc.
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Measurement Bias
• What you designed may not be what respondent sees
• Client’s browser displays the survey based on – platform, browser, monitor, screen/window size
– different users see different survey, e.g., • small screen/window size makes “Next” button not visible
• text doesn’t fit on small window, or requires scrolling for some respondents and not others
• colors, graphics (e.g., visual analog scales) may appear differently
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Non-Completion Bias
• Influenced by– respondent familiarity with web (e.g., click on link)– technical design of survey– bandwidth– convenience (return to finish?)
• Can use mixed-mode surveys to address– e.g., combined web/phone, web/mail
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Subject Recruitment
• Recruitment is biggest bottleneck of clinical research– 30-40% of clinical trial costs – >80% of trials have recruitment delays– 1/20 recruited patients actually enroll
• Web-based recruitment can be international, cheap, fast– e.g., www.stopsmoking.ucsf.edu Dec 05 - Feb 07
• 350,000 hits, 60,000 entered data, 20,000 enrolled• 2/3 Spanish-speaking, 1/3 English• 131,517 visits from 121 countries Jan 12, 05 to April 5,
06
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Threats to Validity
• Selection bias: who is on the web? who isn’t?– digital divide
• Sampling error– non-biased sampling of respondent population
• Non-response bias– enrollees not completing the survey
• Measurement error– poor question wording, variation in how survey
appears on various browsers, non-completion
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Digital DivideInternet Access Broadband Access
<$30,000 41% 8%
$30-49,000 71% 16%
>$50,000 89% 39%
No children 59% 16%
Children in home 76% 29%
White 69% 23%
African-American 56% 15%
Hispanic 48% 14%
"Digital Divide" Still Shapes Media Landscape (10/19/04, Knowledge Networks/SRI); http://www.knowledgenetworks.com/info/press/releases/2004/101904_htmtrends.htm
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Digital Health Divide
• Spanish-language sites have lower quality– 45% of English-language sites vs. 22% with minimal
coverage & complete accuracy (JAMA 2001; 285:2612-2621)
• Broadband more available to higher-income white households with children– uneven potential access to Flash, tele-consultation,
etc.
• Most of divide attributable to income, not to race
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Reducing Sampling Error
• Social sciences and marketing are most advanced in web survey methodology– e.g., Joint Statistical Meetings of the American
Statistical Association
– http://www.knowledgenetworks.com/dmg/index.html
• Recruit a representative sample• Use a pre-assembled representative cohort
Disclosure: I have no relationship with KnowledgeNetworks
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Recruit Representative Sample
• Random digit dialing (RDD) analog equally representative as (land-line) telephone RDD– RDD sampling
– if respondent agrees, provide them with free Internet access (via MSNTV, aka WebTV) or other necessary hardware for duration of participation
– e.g.,http://knowledgenetworks.com/
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Representative Cohorts
• Maintained by e.g., large survey and marketing firms– www.knowledgenetworks.com
• KnowledgePanel is representative of US• can target specific respondents, “response rates of 65-
75%, abandonment rate <2%”
– www.surveysampling.com• panels in 17 countries totaling 3.8 million respondents
– http://experimentcentral.org/ • NSF-funded representative panel for social science
research
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Enrollment Rates
• Response rates typically 30-60%• Affected by
– number of (pre) contacts, whether personalized• most influential factors
– incentives (e.g., Amazon certificate)
– population surveyed, nature of topic, official sponsorship, etc.
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Other Recruitment Methods
• Higher risk of sampling bias– search engines
• need seach engine optimzation (SEO) techniques, e.g., buying Google keywords
– links from related pages– email lists, social networking sites, chat rooms,
newsgroups– other Internet communities
• friends, webrings
– blend traditional and web• give website on radio, TV, print, brochures
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Note on Sample Size
• Estimating sample size– e.g., Google provides traffic history for various
keywords (adwords.google.com)
• Since incremental cost often negligible, less pressure to minimize sample size– not unusal to get large samples (>10,000)
• But high sample size = high accuracy!– may be precise but inaccurate if sample is non-
representative
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Outline
• Clinical Trial Management Systems (CTMS)
– NCI/NIH vision
– case study: UCSF Cancer Center
• Naming data• Running trials on the web• Summary
March 4, 2008: I. Sim Informatics for Clinical ResearchEpi 206 – Medical Informatics
Summary
• Clinical research fragmented, global, essentially separate from clinical care
• Clinical reseach informatics in 2 worlds– most still paper, commercial CTMSs mostly document
centered (PDFs) rather than data or concept-centered
– movement towards caBIG-like world with • standard data elements (CDEs) and case report forms (CRFs)
• common data models (e.g., BRIDG) and interchange exchange standards (CDISC)
• Surveys on web offer promises and methodologic pitfalls