Semantic MediaWiki in Action: CTCAE Revision Project Traci St. Martin, Ann Setser and Lewis Frey Architecture/VCDE Face-to-Face Meeting Northwestern University,

Semantic MediaWiki in Action: CTCAE

Revision Project

Traci St. Martin, Ann Setser and Lewis Frey

Architecture/VCDE Face-to-Face Meeting Northwestern University, Evanston, IL

Oct. 28th, 2008

Agenda

• Introduction to Semantic MediaWiki (2 min)-Traci St. Martin

• Use of Semantic MediaWiki: CTCAE Revision Project (20 min)

• Background of CTCAE Revision Project-Ann Setser

• CTCAE Community Requests for Additional Functionality-Traci St. Martin

• VCDE Input into CTCAE Revision Project-Lewis Frey

• Lessons Learned from CTCAE Revision Project-All (5 min)

Introduction to Semantic MediaWiki

• Semantic MediaWiki (SMW) utilized for BiomedGT

• The Biomedical Grid Terminology is an open, collaboratively developed terminology system for translational research, initially based on NCI Thesaurus

• Goal: Evolve BiomedGT into a set of sub-terminologies that can be:• Federated• Collaboratively developed• Maintained by experts in relevant

research communities• Integrated, where appropriate• With re-use and integration of

existing high quality terminologies

• Additional tools:• Protégé OWL• Collaborative Protégé• LexGrid/LexBIG

Why SMW?

• Traditional approach using NCI Thesaurus didn’t scale• Desire to move to federated terminology development

• Include external groups for wider amount of expertise • Create better acceptance and maintenance of ontologies

• The infrastructure and culture of the semantic wiki is a promising workbench for collaborative ontology engineering.• Easy to learn tool• Open content development platform• Change history management• Formal output rendering (RDF export)

• With some extensions, the SMW can be useful for publishing, annotating and cross-referencing other terminological resources

• SMW offers: • Browsing support (hierarchical view, data elements, etc.)• Authoring support (workflow for proposals, etc.)• Ability to load multiple terminologies

BiomedGT Wiki: Current & Future

• Current Projects within BiomedGT Wiki Environment

• BiomedGT

• Nanotechnology terminology

• Pathology related terminology updating and expansion (interested)

• CTCAE4 Revision project

• Enhancements via collaboration between NCI and Mayo Clinic for BiomedGT Wiki

• Enabling Wiki features Mayo developed for ICD11

• Adding additional namespaces (RadLex, etc.)

• Supporting BiomedGT development

• Maintaining and developing Wiki for CTCAE Revision project

Agenda




• CTCAE Community Requests for additional functionality-Traci St. Martin



CTCAE Background

The Cancer Therapy Evaluation Program (CTEP) of the NCI developed the original Common Toxicity Criteria (CTC) in 1983 to aid in the documentation and analysis of adverse effects of chemotherapy.

The CTC is a list of Adverse Event (AE) terms commonly encountered in oncology accompanied by a grading (severity) scale for each AE.

1983 CTC v1 ~ 50 terms

1998 CTC v2 ~ 300 terms

2003 CTCAE v3 ~ 1000 terms

2008 Draft 1 ~ 820 terms – MedDRA terms

Medical Dictionary for Regulatory Activities – MedDRA

• Is a clinically validated international medical terminology used by regulatory authorities and the regulated biopharmaceutical industry throughout the entire regulatory process, from pre-marketing to post-marketing activities, and for data entry, retrieval, evaluation, and presentation.

• Is the adverse event classification dictionary endorsed by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH).

• Is used in the US, European Union, and Japan. Its use is currently mandated in Europe and Japan for safety reporting.

http://en.wikipedia.org/wiki/Medical_terminology

http://en.wikipedia.org/wiki/Adverse_event

http://en.wikipedia.org/wiki/International_Conference_on_Harmonisation_of_Technical_Requirements_for_Registration_of_Pharmaceuticals_for_Human_Use




http://en.wikipedia.org/wiki/US

http://en.wikipedia.org/wiki/European_Union

http://en.wikipedia.org/wiki/Japan

MedDRA Structure

System, Organ, Class SOC Highest level of the terminology, and distinguished by anatomical or physiological system, etiology, or purpose

26

High Level Group Term HLGT Subordinate to SOC, supraordinate descriptor for one or more HLTs

332

High Level Term HLT Subordinate to HLGT, superaordinate descriptor for one or more PTs

1,688

Preferred Term PT Represents a single medical concept 18,075

Lowest Level Term LLT Lowest level of the terminology, related to a single PT as a synonym, lexical variant, or quasi-synonym (Note: All PTs have an identical LLT)

66,135

• MedDRA translationsCzech Dutch EnglishDutch French GermanEnglish Italian JapanesePortuguese Spanish

• MedDRA includes Standardised MedDRA Queries (SMQs) -groupings of terms that relate to a defined medical condition or area of interest.

• http://www.meddramsso.com

CTCAE v3.0 & MedDRA Use

• CTCAE & MedDRA• MedDRA is used by the biopharmaceutical industry and regulatory

agencies within the ICH regions

• CTCAE use by Industry• CTCAE is widely used in oncology and HIV clinical research

11

MedDRA

List of terms

>86,000

Content: Comprehensive

---

Hierarchy: 5 levels

Medically validated

CTCAE v3.0

List of terms

1,059

Content: Oncology

Severity Scale

Hierarchy: 2 levels

---

CTCAE v3.0 Issues & MedDRA

• Oncology community reported concerns/challenges using CTCAE vs/and MedDRA

• 2006 MedDRA MSSO International Meeting

• 2006 + CTEP, NCI & FDA discussions

• 2007 caBIG VCDE Review of CTCAE

• CTCAE – MedDRA mapping imprecise

• CTCAE terms too general – loss of granularity

• Critical concepts in Grades

• Lack of CTCAE maintenance

• Recommendations

• To facilitate data exchange within internal databases using MedDRA and with regulatory authorities for the purpose of SAE reporting, must establish a mechanism to ‘translate’ or ‘convert’ CTCAE terms from investigators to MedDRA terms.

Rationale for CTCAE v3.0 Revision

• CTCAE v3.0 Terms• Multiple concepts• One/many element of Grade description is critical AE concept• Not all MedDRA terms

• CTCAE v3.0 Revision Goals• Harmonize with MedDRA

• 100% MedDRA single concepts• Revise and extend AE terms and severity indicators to be machine interpretable;

conformance with caBIG Vocabulary Standards• Establish a formal, ongoing governance for future versions

• Understandability, Reproducibility, Usability

CTCAE v3.0 Multiple Concepts in one AE Term

14

Mapped to MedDRA: Fatigue 10016256

CTCAE v4.0/MedDRA– Fatigue– Asthenia– Lethargy– Malaise

MedDRA Preferred TermFatigue + 20 LLTs

16

CTCAE v3.0 RevisionCritical concept listed in Grade only – not as AE Term

CTCAE v4.0/MedDRA– Allergic reaction– Anaphylaxis

CTCAE Governance GroupFDA,CBIIT,CTEP, PhRMA

Steering committeeCBIIT, CTEP,VCDE, WG Leads

CTCAE v3.0 Revision Project

Semantic Media Wiki

CTCAE Editing/modeling

Establish long term governance of CTCAE

Develop the strategic vision of CTCAE Drive the development of CTCAE

Defines work; prioritizes and schedules the content Review/

Comments

CTCAE Version 4.0

Internal review

WGWG

WGWG

WGWG

WGCTCAE draft version

Working Groups / SOCs

CTCAE v3.0 Revision ProjectMethods – Wiki

Agenda







CTCAE Iteration 1

• Publishing CTCAE v3.0:

• CTCAE v3.0 existed as an Excel spreadsheet.

• Created a tool to map the spreadsheet to LexGrid model

• Once in the model, the tool generated XML in LexGrid format

• This XML was loaded into LexGrid database using LexBig API

• Used existing LexWiki tab plug-in to Protégé to get the LexGrid contents to load into Protégé database and publish to Wiki

CTCAE3 spreadsheet

Map to LexGrid model

XML Generated

XML loaded to LexGrid DB

Publish to WikiLexGrid to Protégé DB

Protégé Plug-in

CTCAE Iteration 1

CTCAE Iteration 1

• Utilized additional SMW extensions, created new Semantic Forms, and created new Templates specifically for CTCAE.

CTCAE Iteration 1 Functionality

• Initial requirement - to display the data in a tabular format.

• Utilized Dynamic Page List extension to dynamically show all the Adverse Event terms of a Category in a table.

• Query looks for all pages in a Category and displays them in a table . The Adverse Event (AE) name displays as a link instead of the URL


• Initial requirement continued:

• Created templates to display original content and proposal in a tabular format.


• Using features of Semantic Forms, created a proposal mechanism with a drop down menu for users to select Proposal status of Edit, Split or New Term.


• Utilized Header tabs extension in SMW to easily toggle between the Tabular format and the Factbox.

CTCAE Iteration 1: Processing proposals

• Created mappings based on rules for Edit, New, and Split Term status.

• Used RDF feed to read the proposals and render to LexGrid model.

• Used RDF feed and created a script to generate spreadsheets for Ann to review proposals, MedDRA review and reporting to Steering Committee.

• Published to Wiki using same initial process.

• Published to a new CTCAE SOC Draft 1 namespace for 2nd Iteration of CTCAE revisions.

• Enabled referencing CTCAE 3 proposals.

CTCAE Iteration 1: Usability enhancements

• Based on user difficulties with standard Talk/Discussion page, utilized Article comments to change to a more user-friendly format.

Old formatNew format

CTCAE Iteration 1: Usability enhancements

• Users experienced difficulties with using the Edit tab instead of creating a proposal.

• PHP code modified to hide the Edit tab for non-Sysop users.

Edit tab hidden

CTCAE Iteration 2: Changes

• Iteration 2 organizes data by SOC instead of Category

• Established mapping between Category, Adverse Event term, and SOC. • Allowed publishing by SOC but maintains Category information in database.


• Created a new Header tab and link back to CTCAE 3.0 Adverse Event term and Proposal (from which the CTCAE Draft term originated).

• Link opens in a new tab or window• CTCAE Category (v3) is ‘locked’(via a change to the CTCAE form).

• Users cannot edit 1st iteration proposals.


• Modified Status

• Allowed for separate status for Adverse Event & Grades

• Created new options: No Change, New, Edit, Delete

CTCAE Iteration 2: Additional Enhancement

• Customized Auto-complete feature

• Use Ctrl-Space or enable in My Preferences

• Easier to find specific article (AE term)

Agenda





• Demonstration of Media Wiki-Traci St. Martin



CTCAE VCDE Subgroup Scope

Propose governance and structure for the CTCAE caBIG terminology.

• Proposals that are designed to help CTCAE pass the VCDE vocabulary review for terminologies that support value domains within the caBIG infrastructure

• Support CTEP’s need for multiple formats of CTCAE such as Excel and PDF as well as a paper version of the terminology

• support for use in clinical settings consistent with the way it is currently used

• Subgroup: Salvatore Mungal, Mike Riben, Stuart Turner, Lewis Frey & Larry Wright

CTCAE V3.0 PDF Format

Agenda




• CTCAE Community Requests for additional functionality-Traci St. martin

• Demonstration of Media Wiki-Traci St. Martin



Lessons Learned

• Overall – good learning experience, great user community leadership!

• Modified Development process important for quicker deployments to react to user needs

• Stage is “Production”

• Developed build script to make deployments easier

• Quick response important for new users

• Encountered slowness due to maintenance by infrastructure team

• Maintenance schedule modified to accommodate user availability needs

• Workflow

• Fully understanding workflow early on is important for determining features needed for users and for processing proposals

• Workflow functionality not developed for CTCAE which necessitated developing scripts to generate spreadsheets for reviewing data

Lessons Learned

• Requirements defined before NCI clinical team understood Wiki

• ‘Maintain paper format’

• Reluctant users – clinicians

• Performance – very slow at times

• Data in – nothing out

• No summary data

• No analysis of complete vs. to-do

• Data quality poor

• Misspellings; Upper / lower case errors

• Non-MedDRA terms

• Abbreviations

• Inconsistent use of indicators – Edit, Split, New

• Identified need for additional indicator – Delete

• Lack of ‘checks’ associated with indicators

Next Phase of Wiki Work

• Training for WG members

• Well-defined instructions

• Strive for consistent use of indicators – Edit, Delete, New

• Accurate data entry (spelling, case, no abbreviations, etc.)

• New terms must be pre-approved with MedDRA

• Performance – ?

• Data in – Wiki team working on provision for summary/report data

Documents

Semantic MediaWiki in Action: CTCAE Revision Project Traci St. Martin, Ann Setser and Lewis Frey Architecture/VCDE Face-to-Face Meeting Northwestern University,