March 8, 2005: I. Sim Research Informatics Epi 206 – Medical Informatics Ida Sim, MD, PhD March 8, 2005 Division of General Internal Medicine, and Program

March 8, 2005: I. Sim Research InformaticsEpi 206 – Medical Informatics

Ida Sim, MD, PhD

March 8, 2005

Division of General Internal Medicine, and Program in Biological and Medical Informatics

UCSF

Clinical Research Informatics

Copyright Ida Sim, 2005. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.


Guest Lecture Next Week

• Paul Tang, MD Chief Information Officer, Palo Alto Medical Foundation– Tuesday, March 15, 8:45 to 10:15 am– state of the art electronic medical record

• same one that Kaiser is spending $2.8 billion on

– the promise and the reality

• Last class lecture at 1-2:30 Tuesday March 15


Outline

• “Critical Crossroads”• Biomedical informatics base for clinical

research– design and execution– management and analysis– application to clinical care

• Summary


Crossroads for Clinical Research

• 1st translation– from basic science to clinical science

• 2nd translation– from clinical science to clinical practice


Clinical Research in America• seen “…for what it is—a fragmented cottage industry

constituted of multiple stakeholders, some of whose funding is derived from governmental sources and others from industry, with no overarching vision, no cohesive organizational framework, and at times not even a common forum for dialogue or active collaboration. The current poorly articulated and highly compartmentalized components of the existing nonsystem are inefficient and often redundant. Hence, they diminish effectiveness and increase costs of translating basic research to patient care while often not contributing materially to its safety or efficiency.

• Most importantly, this existing but outdated infrastructure, put in place nearly 40 years ago to support a much narrower and less professional spectrum of clinical research activities, is currently functioning on overload. The US capacity to translate basic science into improved health care for its population is rapidly being exceeded by the burgeoning scientific opportun-ities at hand. Basically, the ‘clinical research grid’ is failing."

(Crowley, et al, JAMA 2004; 291:1120-1126), Institute of Medicine


Properties of Ideal Research Grid?• Integrates research design, execution, reporting, and analysis• Integrated with clinical care systems

– for capturing data from routine care– for fielding research decision support modules at point of care

• to increase generalizability (e.g., effectiveness and pragmatic trials) • to allow for greater sample size for geno-phenotype correlations

• Allows reuse and sharing of – information sytems (e.g., data collection forms, decision support,

security features, adverse events reporting, etc)– protocols and protocol elements (e.g., eligibility rule for menopause)– data (e.g., for pooling and meta-analysis, for data discovery)

• Integrated with decision support systems for applying results to clinical care

• Secure, flexible, robust, maintanable, cheap, portable…


National Clinical Research Enterprise

• IOM suggested a public-private partnership to– set and

maintain national standards

– coordinate and fund sustainable infrastructure


NCRE Objective

• Continuously improving national clinical research infrastructure to sustain safe and efficient transfer of basic research to practice

• Agenda– increasing public understanding, confidence,

participation in clinical research– adequately trained, diverse workforce– improving the US biomedical informatics base for

clinical investigation– funding this innovative clinical research enterprise

• Lots of activity in 2003, nothing since…


Outline



• Summary


Biomedical Informatics Base — 3 Spheres

Administrative Clinical Care Research

ClinicalBilling

Physical Networking

Standard Communications Protocols (e.g., HL-7)

Standard Vocabulary

PracticeManagement

Systems

ElectronicMedicalRecord

??

Medical BusinessData Model

Clinical CareData Model

??


Industry is the Innovator

• RCTs was a $3.6 billion business in 2000 (C. Scott, 7/00)

– in 1988, 95% of RCTs conducted by academics– now, over 80% conducted by industry

• Ergo, much of the technology innovation in clinical research execution is going on in industry– Applied Clinical Trials software directory

• http://www.actmagazine.com/appliedclinicaltrials/article/articleDetail.jsp?id=145986

• NIH just getting into this with Roadmap initiative– National Electronic Clinical Trials Research Network

(NECTAR)– NCI caBIO also very involved in clinical trial informatics


The Clinical Trial Cycle (per NCI)

New Ideas

Protocol & Funding

Findings

Approval &Preparation

Design Trial

Activate TrialConductTrial

UtilizeResults •trial simulators

•trial costing•protocol authoring

•IRB approval•CRF design

•data management•remote data entry•GCP compliance

•data analysis•reporting


Infrastructure for Clinical Trials

New Ideas

Protocol & Funding

Findings


Design Trial

Activate TrialConductTrial

UtilizeResults •trial simulators

•trial costing•protocol authoring

•IRB approval•CRF design

•data management•remote data entry•GCP compliance

•data analysis•reporting

• a few companies• many companies

• FDA electronic submission standards

• a few companies


Research and Care Together

for patientcare & policy

for clinical researchNew Ideas

Protocol & Funding

Findings


DesignStudy

ActivateStudy

ConductStudy

UtilizeResults for basic research

ClinicalCare


Joint Infrastructure for Care and Research


ClinicalBilling

Physical Networking


Standard Vocabulary

PracticeManagement

Systems

ElectronicMedicalRecord

Clinical Research Management

Systems



Clinical StudyData Models


Outline



• Summary


Running a Study• You are planning a study on infant jaundice...• What relevant studies have been completed on this

topic?• What ongoing studies should you know about?• You’re interested in running your study over the

web as much as possible– what types of study activities can be done over the

web?– how good is the technology for these activities?


Relevant Trials: Completed/Ongoing• Published

– Medline– Cochrane Controlled Trials Register

• ~328,000 records of controlled trials• manual logging of CCTs by hand searching journals• accessible from UCSF machines (IP address) only

• Non-profit/government– www.clinicaltrials.gov

• 12,600 trials, ~3000 open• NIH-supported and some commercial cancer and AIDS trials

– www.cancer.gov/clinicaltrials– www.actis.org

• AIDS Clinical Trials Information Service

http://www.updateusa.com/clib/CLIBINET.EXE?A=1&U=&P=

http://www.controlled-trials.com/


Trial Registries, cont.

• Commercial: mostly for patient recruitment– www.centerwatch.com– www.ClinicalTrialFinder.com– www.controlled-trials.com– www.clinicaltrials.com– etc., etc., etc.

• Meta-registries– www.trialscentral.org (from Cochrane people)

• pointers to hundreds of clinical trial registries, by disease– metaRegister of Controlled Trials

• >14,000 commercial and ongoing trials


Case: Clinical Research Informatics• You are planning on a study on infant jaundice...• What relevant studies have been completed on this

topic?• What ongoing studies should you know about?• You’re interested in running your study over the

web as much as possible.– what types of study activities can be done over the

web?– how good is the technology for these activities?


Is the Promise Realized Yet?

• “The future appears bright for web-based clinical trials.” – Marks, et al, Paradigm shifts in clinical trials enabled

by information technology. Statist Med 2001• “Valid reasons exist for the slow growth of

technology adoptions in clinical trial activities”… “These efficiency gains are only possible as attitudes change regarding moving from old inefficient manual processes to more automated processes”– Marks, Validating electronic source data in clinical

trials, Control Clin Trials 2004


Clinical Study Tasks

• Project website• Subject recruitment• Eligibility determination• Protocol and forms

– distribution– online data entry

• Randomization• Data collection


Project Website

• Project website of e-text pages– GISSI website GISSI-x summaries, references

• http://wwwcardio.marionegri.it/trials.htm

– HERS main results revised tables from JAMA report• http://www.keeptrack.ucsf.edu/hers2/HERSfindat.htm

• Requirements– web server computer

• use a web hosting service (see http://www.cnet.com/)• or have a web server program (e.g., Apache)

– pages of material• produce these using a web editor (FrontPage, Dreamweaver)

• or use Word (save as HTML file)


JIFE Client/Server Model

itsa

jaundice

ucsf.edu

KaiserOakland

KaiserSanta Clara

Internet

at homedial-in to itsa.ucsf.edu via modem

pacbell.net

aol.com

LAN

KaiserSan Diego

• The “jaundice.ucsf.edu” computer hasweb server software. It “serves” web pagesin response to http commands such ashttp://jaundice.ucsf.edu/project-home.html


• Personnel– webmaster: handles the machine stuff– web designer: produces text & graphical content

• Status: easily doable today• More difficult if wish to export static views of

Access database to web– e.g., go to website that shows latest contents of a data

table– requires more advanced database/web programming

Project Website (cont.)


Exporting Access Data to Website• Static views of Access database on web

– ASP program tells server how to make web pages of data in the database– web server must be able to “serve” ASP pages

Access JIFE DB

Internet Information Server 4.0

KaiserOakland

KaiserSanta Clara

Internet

KaiserSan Diego

Active Server Pages (ASP)

jaundice.ucsf.edu


Automated Eligibility Determination

• Study enrollment is big bottleneck• Eligible patients: patients whose characteristics

match with eligibility criteria• For computerized matching, need to have computer-

interpretable descriptions of– patient characteristics– the eligibility criteria

Match Eligible Patients

EHR

Eligibility Rule


Eligibility Example• Eligibility criterion: women who are 2 or fewer

years post-menopause, as defined in NCI’s Common Data Elements set

• Allowed values:Above categories not applicable AND Age < 50Above categories not applicable AND Age >=50Post (Prior bilateral ovariectomy, OR >12 mo since LMP with

no prior hysterectomy and not currently receiving therapy with LH-RH analogs [eg. Zolades])

Post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy)

Pre (<6 mo since LMP AND no prior bilateral ovariectomy, AND not on estrogen replacement)


EHR Data Needed

• Gender• Age• Time since LMP, whether

– 6 or fewer months, or 12 or more months

• Past surgical history– bilateral ovariectomy and/or hysterectomy

• Therapy– LH-RH analogs, or– estrogen replacement


Computer-Interpretable Eligibility Rule

• Logical rules– (Prior bilateral ovariectomy) OR (>12 mo since LMP AND no prior hysterectomy)– first order logic is the best representation model for this

• Temporal constraints – greater than 12 months since LMP...– representing time requires second-order logic

• Can do simple cases with database rules and triggers– new, “temporal” SQL allows time-based queries on

intervals, timepoints, etc.


Promising, but...

• Coding of eligibility rules is difficult• Richly detailed, coded EHR not widely available

or well integrated• At present, can only expect computer to suggest

potential subjects. EHR can– prompt MD in real-time to refer patient to study, or– periodically batch notify MD of eligible patients, or– send letter of solicitation to patients

• Similar problems bedevil automated identification of guideline eligibility


Protocol and Forms Distribution• Allows for centralized forms management and storage

through a project website• If expecting users to download, print, fill out and fax form

back– need protocol and forms in electronic format (e.g.,Word

or PDF)– for forms not available in electronic form

• scan it using a scanner ($100-$4000)– makes an image of the page (e.g., .gif or .jpeg)

• optical character recognition (OCR) scanning– convert scanned text into an editable document

(e.g., Word)• Status: easily doable today


JIFE Forms Download

itsa

jaundice

ucsf.edu

KaiserOakland

KaiserSanta Clara

Internet


pacbell.net

aol.com

LAN

KaiserSan Diego

• “jaundice.ucsf.edu” “serves” forms such ashttp://jaundice.ucsf.edu/case-form.pdf for printing out


Protocol and Forms Distribution

• If expecting users to enter data online over the web– need someone to design the forms and build them to be

served over the web• e.g., using Access Visual Basic• more advanced: use combination of Access and FrontPage

– need security mechanisms (e.g., user login)– need data validation checks built into forms entry– data forms must send data to a database

• needs ODBC (Open DataBase Connectivity) drivers, etc.

• Status: doable with more advanced programming


Infant Jaundice Online Forms

itsa

jaundice

ucsf.edu

KaiserOakland

KaiserSanta Clara

Internet


pacbell.net

aol.com

LAN

KaiserSan Diego

• “jaundice.ucsf.edu” “serves” online entry formssuch as http://jaundice.ucsf.edu/case-form.asp.Users enter data, which get checked at the clientside, and data is sent back to “jaundice.ucsf.edu.”


• Requirements– a web-based data collection form to collect patient information– programs to verify eligibility and randomize patient– program to generate a response to the enroller

– security, privacy, and backup provisions• Some commercial systems do this for you• Status: doable with some programming

Web-based Randomization

Project Central Enroller

patient info

randomization results


Electronic Data Capture• Fax• Voice

– most systems ~ 95-99% accurate for restricted domains– some systems need to train computer to your voice

• Handhelds/tablet computers– need docking station for downloading– wireless PDA/tablet computer can’t be far from a receiver

node• radiofrequency: slow, prone to interference• infrared: requires line of sight between PDA and node

– must be HIPAA-compliant transmission• Remote physiologic monitoring


Issues in Remote Data Capture

• Managing copies (local and central copies?)• Concurrent updates

– what if 2 people want to update same record?

• Merging data• Security and privacy

– HIPAA makes this all much more complicated

• System downtime• System response time• Workflow issues


Design and Execution Summary

• Project website– easily doable today

• Automated eligibility– partially doable today; fundamental problems unsolved

• Protocol and forms distribution– easily doable today

• Online data entry– doable with more programming

• Randomization– doable with some programming

• Remote data capture– doable, HIPAA making this trickier, more expensive


Trials on the Web Summary

• Held back by paper-based processes and– FDA requirements (for electronic documents not data)– lack of mobile hardware– attitude (e.g., on data validation of Case Report Forms)

• Solutions available, but not integrated• NIH NECTAR initiative just starting• UCSF considering an Informatics Center with

centralized services


Outline



• Summary


Case Study: RCTs

• How many Phase III RCTs does NIH fund?• What percentage of depression RCTs are placebo-

controlled?• What percentage of patients in CHF trials have

renal failure?• Which acute MI trials report 5 year mortality?• Is industry funding correlated with positive

outcomes?• What percentage of trials report outcome results

selectively?


NIH Data Reporting Requirement• As of May, 2003

– all grants over $500,000 direct required to “report all data publicly”

– variation among institutes on how to implement this

• no standard format

• What use is “the data” if the study design is poorly reported, not computable, or not available?


Summary on Management/Analysis• Systematic under-investment in informatics for clinical

research results management and analysis– need common data structures for various study types (e.g., RCTs,

systematic reviews, guidelines)– need standardized research variables (e.g., NCI Common Data

Elements)– need standardized coding of variables and covariates

• Contrast to rise of computational biology/bioinformatics science– DNA sequences: GenBank; metabolic pathways: BioCyc– pharmacogenetics, pharmacogenomics: PharmGKB– protein sequence: UniProt; protein structure: PDB, etc.


Outline



• Summary


2nd Translation, to Clinical Care

• 2 approaches using information technology– “firehose” approach

• if 1 article/guideline is good, 1000 is better…

– decision support to present evidence-based action options


IT for Translation• “Non-thinking” information technology approaches

– evidence-based medicine resources• ACP Journal Club, Clinical Evidence, PubMed, Up-to-Date• Cochrane systematic reviews

– guidelines (over 3500 guidelines)• www.guidelines.gov

– quality measures (301 measures)• www.qualitymeasures.ahrq.gov

• Clinical decision support systems– … in which the characteristics of an individual patient

are matched to a computerized clinical knowledge base for patient-specific recommendations

– need both coded EHR and coded biomedical knowledge


Computable Biomedical Knowledge Bases

• Pharmacogenetics/genomics: PharmGKB• SNPs: SNP Consortium• Randomized trials

– Trial Bank Project (http://rctbank.ucsf.edu/Presenter/)• computable database of RCT design, execution, results

• Physiological models– Archimedes: diabetes

• No computable repositories of – clinical guidelines– systematic reviews, decision/cost-effectiveness models– “textbook” information


Shared Knowledge Bases (KBs)• Should share the primary knowledge (e.g., RCT results)!

– open access knowledge repository– each system can build its own rule set based on shared

evidence– KB updating and maintenance also shared

• E.g., Global Trial Bank partnership w/ Public Library of Science– computable RCT protocols and results tied in to trial registration

(e.g., clinicaltrials.gov)– peer reviewed by PLoS Clinical Trials– open access knowledge base and journal, with an open peer

review/discussion forum– decision support systems/EHRs can directly query GTB


Summary: Application to Care• Currently at first generation

– static web/e-text information resources• humans have to retrieve and read articles

– individual decision support systems have expensive, hard-to-maintain, hand-coded knowledge bases

• To move to next generation, need– widely deployed EHRs, reasonably coded in SNOMED– shared computable biomedical knowledge bases– a better theory of when and how to translate evidence

to care


Outline



• Summary


Joint Infrastructure for Care and Research


ClinicalBilling

Physical Networking


Standard Vocabulary

PracticeManagement

Systems

EHRExecutionAnalysis



Clinical StudyData Models

Application to Care


Summary

• Clinical research grid is failing– concerted action is required

• Need to integrate solutions across practice management, clinical care, clinical research

• Informatics is a critical component of solution– design and execution: e.g., NECTAR initiative– computable knowledge bases of clinical research

studies: very, very early in this– application to clinical care: need computable EHR and

biomedical knowledge


Teaching Points• Many aspects of clinical research can now be done

electronically, but in fragmented fashion– limitations include reliance on paper, lack of wide EHR

adoption, lack of standard coding

• Research needs to be more integrated with care– for sample size reasons for geno-pheno correlations– for effectiveness, pragmatic trials

• Informatics infrastructure needs to be shared• The dialogue is just beginning…

Documents

March 8, 2005: I. Sim Research Informatics Epi 206 – Medical Informatics Ida Sim, MD, PhD March 8, 2005 Division of General Internal Medicine, and Program