Got Data? Moving Ahead Toward Network

Data Collection and Dissemination

Elizabeth M. Z. FarmerBrian McCourtPatrick Loebs

National Child Traumatic Stress NetworkAll Network Meeting

San Diego, CADecember 10-13, 2003

What are the Data Core and The Core Data Set?

• Data Core– One of the five cores within the Network– Mission: To provide oversight and guidance in the

design, collection, and analysis of Network data– Facilitate, support, and coordinate data collection

efforts by the National Center and by other cores, committees, task forces, working groups, etc.

– Betsy Farmer (Duke) is the Chair. Includes members from Duke (including Duke Clinical Research Institute) and UCLA

– Includes 2 current committees:• Data Operations Committee

– Assists and advises Data Core on collection, management, and dissemination of Network Data

• Measures Committee– Assist Network in planning and implementing clinical

and service measurements needed to accomplish Network goals

What are the Data Core and Core Data Set?

(continued)

• Core Data Set– Standardized set of domains and measures to be

collected across Network sites– Only Network-wide data collection that will

provide answers to central Network questions:• Whom are we serving?• What types of problems, symptoms, needs do youth have?• What types of trauma have youth experienced?• What types of treatment are we providing?• To what extent and in what ways do youth improve during

treatment?

– Absolutely essential for ensuring that the work being done within the Network is systematically measured, disseminated, and recognized

Overview of Today’s Session

• Background, rationale, and current status of the Core Data Set– Betsy Farmer

• Overview of Issues related to Implementation of the Core Data Set (Technology and Technicalities)– Brian McCourt

• Overview of regulatory processes toward Core Data Set implementation– Patrick Loebs

• Discussion, questions, comments, suggestions– Everyone

Background for the Core Data Set

• From the very beginning (i.e., summer of 2001), there was an expectation that the Network would develop and implement a core data set– Who are we serving? – How are we serving them?– Is it making a difference?

• In September 2002, the Data Operations Committee was formed and charged with developing the Core Data Set– Became the primary objective of the Data Operations Committee– Initial meeting with full committee plus representatives from

SAMHSA– Unlike many other committees, Data Operations was given a short

timeline to develop the components of a Core Data Set (by start of 2003)

Priorities and Components of the Core Data Set

• Priorities– Content Areas:

• Client Characteristics• Outcomes• Service Use

– Principles for selecting domains/measures• Needed to be as short as possible (to fit into existing

practice)• Focus on CORE domains (what was absolutely

necessary, not just what would be nice to know)• Needed to be systematic and consistent across sites

and clients• Preference for psychometrically sound assessments that

had demonstrated properties, widespread use, and applicability across diverse populations

• Attempt to include different approaches to assessment to meet needs of the Network and key stakeholders

Components of the Core Data Set

• Clinical Characteristics Form– Client demographics– Current living situation– Insurance– Severity of problems (e.g., ‘real world’ functioning)– Current service use– Trauma history – Primary presenting problem/focus of treatment

• Outcomes– Child Behavior Checklist (CBCL)– Trauma Symptom Checklist for Children (TSCC-A)– PTSD Reaction Index

Proposed Approach to Collecting Core Data Set Information

• Core Data Set is designed to provide data on the majority of youth served by the Network

• Designed to be collected on youth receiving clinical services– i.e., not those seen at drop-in centers, served via referrals, by ‘first

responders,’ etc.

• Will collect data on youth who begin receiving services once the Core Data Set is implemented – i.e., we will not pick up all active cases or attempt to gather retrospective

data

• Should be collected on all eligible youth at least twice– i.e., baseline and follow-up

• Data collection, entry, scoring/feedback, and transmission will be aided by web-based data capture program

Timeline for the Core Data Set

• ASAP

• Necessary steps prior to implementation– Clearance by Office of Management and Budgets (OMB)– Information Sharing and Training with Sites– Finalization of web-based data capture program– Completion of regulatory compliance (i.e., IRB, HIPAA)

• Realistic Timeframe– Ready to fully implement by Spring/Summer – Information Sharing and Regulatory work completed by

end of Winter– Training completed by Spring– Currently ready to work with any sites that are ready to

try implementing part/all of the Core Data Set

QI Initiative: Success StoryCRUSADE: Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes with Early

Implementation of the ACC/AHA Guidelines

Technology & Technicalities

• Thoughts from 1st ANM, February 2002

• Background

• Current Plans– Web-based Electronic Data Capture (EDC)– Data Transfers

• Next Steps

Assumption:

Centers have different data management infrastructure and systems

Option 1: DCF (Paper) based option

Option 2: Remote (Internet) based option

Option 3: Distributed data management option

Option 4: Hybrid (Remote + Distributed) option

Conclusion:Define data to be collected and better understand

Center capabilities

Data Aggregation Approaches1st ANM, Los Angeles, 2/20/2002

Implementation Survey

Key Survey Results (n=36)

• In all domains, majority of Centers capture that data on paper

• All Centers have internet access, 29 have high speed• Some Sites indicated a preference for electronic

transfers:– Demographic Data: 9 out of 36 Centers– All other domains: 2-6 out of 36 Centers

• SAS, SPSS, Access & text formats suggested

Web-Based Data Management

Process

• Secure website• Collect Clinical Characteristics and Assessments• Interactive error checking during entry• Assessment Clinical Scoring done centrally

(Process TBD)

Data Transfers

Current Thinking

• The Data Core plans to pursue a data transfer process for receiving the Core Data Set in addition to an EDC process

• A standard format & process will be used for all Centers• Exact specifications and availability will be available

after EDC process is rolled-out• Input/Collaboration with Centers on process & format is

planned

Data Transfers (continued)

Process

• Secure FTP• Data files plus descriptive summary (i.e. record counts)• Automated error checking during load/merging• Automated data quality checks and feedback (missing,

inconsistent, unexpected data) same as EDC data• Format and data updates performed by originating Center

Pipe delimited text file, 1 record per client2099|123456|12/09/2003|0|0|6|0|0|1|0|0|1|NEIGHBOR|99

VISIDT

PREVEPIS<I:3>

FIRSTVIS<I:3>

EPINUMS<I:3>

PARENT<I:3>

OTHADULT<I:3>FOSPARNT<I:3>

AGENCY<I:3>

CHILD<I:3>FROMSP<I:3> OTHERSP<V:200>

GUARDIAN<I:3>

GUARDSP<V:200>

DEMOG<1 REC/PT>CNTR <I:4>PATID <V:20>

Data Transfers (continued)

Issues to Consider

• Different clinical data definitions• Not enough Centers (or client data) support

implementation• Unable to use standard transfer format or process• Weigh impacts on supporting most sites and getting most

data possible across all data collection options

Data Transfers (continued)

Next Steps

• Already have electronic data and export mechanism: – Evaluate if your data definitions (clinical, not technical!)

match the Core Data Set• Implementing electronic data system:

– Align data definitions, consider aligning data structure– Will need export mechanism

• Tell Data Core (Becky Warlick or Brian McCourt) if your Center is interested

• Related learning opportunity: Electronic Clinical Data Discussion Session ?

Moving Ahead toward Network Data Collection and Dissemination

Regulatory Processes - Presentation Overview

– Process for Defining this Data Collection Effort – Quality Improvement Initiative or Research?

– Example of QI vs. Research “Decision Tree”– Approval - List of IRB written assurances required for data

collection initiative– List of IRB submission documents to be provided to

Centers

“Defining the Data Collection Effort - Discussions and Interpretations”

• Discussions within the Data Core• Consultation with Data Core Committee Members• IRB and Regulatory Consultation• “Like” Projects at Duke as models?• Federal Guidance’s/Code of Federal Regulations• “Common Rule” (45 CFR Part 46) • QI versus Research

“The Common Rule” –Interpretations and Decisions

• The Common Rule (45 CFR Part 46)

– Research: a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for other purposes. For example, some demonstration and service programs may include research activities.

“QI vs. Research; Decision Tree”

– Article in JAMA (2000) by Caserat et. al*, describes a Quality Improvement Initiative as “cycles of interventions that are linked to assessment that have the goal of improving the process, outcome and efficiency of care.”

– Design and Goal of this data collection initiative correspond to this definition.

*Caserat et al., J Am Med Assoc. 2000; 283: 2275-2280.

“QI vs. Research; Decision Tree” (continued)

• Two More Distinguishing Questions In JAMA Article regarding QI vs. Research

– Question 1: Are the majority of patients expected to benefit from the knowledge to be gained?

– Yes, this initiative will collect information pertaining to interventions that are linked to assessment and that have a goal of improving the process, outcome and efficiency of care to traumatized children and their families.

“QI vs. Research; Decision Tree” (continued)

– Question 2: Will making the results generalizable require any additional risks or burdens to the patient?

– No, this initiative does not dictate or require any treatment; it is not a clinical trial. It involves “minimal risk” (45 CFR Part 46). Also, this initiative will seek to collect information normally collected as part of standard clinical practice across network Centers.

“QI vs. Research; Decision Tree” (continued)

• Some Centers/IRBs may not view this data collection initiative as QI.

– However, the data collection initiative, if deemed research, may qualify for “waiver of authorization”.

– That is, under HIPAA, PHI can be used without a subject’s authorization if the information is stripped of identifiers (e.g., name, Social Security number, medical record number, dates, etc.)

• Therefore, HIPAA Rules will be followed by Data Core– Data Use Agreement will be signed by Duke and

Centers– Process for receiving Limited Data Set (LDS)

from Centers will be in place• “De-identified”• “Mirror database” and link field (code

allowing “road’ back to original database) created by Data Core

• Analysis database becomes “anonymized”

“QI vs. Research; Decision Tree” (continued)

“QI vs. Research”IRB List of Assurances

• Data Use Agreement (DUA)– Initiated by Duke Legal– Signed by Center and Duke (limited data set

[LDS] recipient)– Copies retained at Center and Duke

• “Crosswalk”/client log (a template form)– Distributed to Centers by Data Core– Filled out by Center staff

• Retained at the Center in a safe and secure area.

– Data Core is not allowed to view this log nor will it have possession of it

“QI vs. Research”IRB List of Assurances (continued)

• Documentation from Centers– When LDS sent to Data Core: no client

information has been transferred, nor will it be– Copies of documentation retained by Data Core

• Documentation from Data Core– When receiving LDS Center, no client

information has been received, nor will it be requested.

– Data Core will document that all data will be de-identified and anonymized upon receipt.

“QI vs. Research”IRB List of Assurances (continued)

• Documentation from Center– IRB document(s) stating whether or not this

initiative was viewed as a QI or R– Copies of this documentation retained by

Data Core and sent to the Duke IRB (e.g., audit purposes)

NOTE: All documentation will be collected, tracked and maintained by Data Core and will be retained in the Regulatory Files for the Project

“QI vs. Research” – Documents to be Provided

• IRB Documents to be Provided to Centers– Initiative/Program Summary– QI versus Research Position Paper– Data Use Agreement– Client Log (“Crosswalk”) template– Core Data Set Form with Assessments– Information/links on MPA/FWA requirements– ICF Templates (if requested)– CD with electronic versions of above