Download pptx - Automatic generation of MedDRA terms groupings using an ontology

Automatic generation of MedDRA termsgroupings using an ontology

Gunnar DECLERCKa, Cédric BOUSQUETa,b and Marie-Christine JAULENTa

a INSERM, UMRS 872 EQ20, Université Paris Descartes, France.b Department of Public Health, CHU University of Saint Etienne, France.

MIE 2012, August 27th, Pisa

2

Context and Rationale

MedDRA (Medical Dictionary for Drug Regulatory Activities) : standard terminology used to code adverse drug reactions (ADRs) in safety reports for postmarketing drug surveillance.

Used for Signal detection : Case reports coded with MedDRA stored in databases (i.e. FDA pharmacovigilance Database

for the US, WHO Vigibase for Europe) Data mining algorithms used to find statistical correlations between ADRs (several MedDRA

terms defining a unique medical condition) and drugs (a “signal”) Empirical studies to assess the causal relationship between the drug and ADR.

MedDRA hierarchical structure

WP Work Package title1 Project management and administration

2 Framework for pharmacoepidemiological studies

3 Methods for signal detection

4 New tools for data collection from consumers

5 Benefit-risk integration and representation

6 Validation studies involving an Extended Audience

7 Training and communication

Main goal: To develop and evaluate semantic driven methods for grouping MedDRA terms to improve signal detection performances.

Hypothesis: Signal detection is improved when algorithms use groupings of MedDRA terms referring to the same ADR condition (rather than one single term).Þ Need a way to assist human experts to build MedDRA groupings (currently made manually)

Method: Building an ADR ontology (OntoADR) providing MedDRA terms with formal machine-processable defintions to support automatic MedDRA terms grouping procedures (OWL queries selecting terms on the basis of their semantic properties).

PROTECT WP3-SP6 “Novel techniques for grouping ADRs to improve signal detection”

3

PROTECT. Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium. IMI (Innovative Medicines Initiative) project (2009- 2014).

4

MedDRA terminology enriched with Snomed-CT concepts formal definitions

34994 concepts (20856 MedDRA 13.0 terms / others from Snomed-CT)

26 Snomed-CT relations used to express medical meaning of MedDRA concepts

OntoADR ontology

HASFINDINGSITEspecifies the body site affected by a condition

HASOCCURRENCErefers to the specific period of life during which a condition first occurs

…

5

Hierarchical relations from SOC level to PT (Preferred terms) level are converted to subsomption (subclass_of) relations

LLT (low level terms) are integrated as annotation labels of PT concepts PT level

SOC level

HLT level

HLGT level

Ontologizing MedDRA

6

When it is possible, MedDRA concepts are mapped with Snomed-CT concepts via UMLS metathesaurus

Þ Semantic information describing Snomed-CT concepts used to build the formal definition of MedDRA concepts.

When mapping impossible, formal definition made manually (via collaboration between knowledge engineers and medical experts)

Formal definition of « Dilatation intrahepatic duct congenital » Meddra PT concept after Snomed-CT mapping (mapped with

“Congenital dilatation of lobar intrahepatic bile duct”)

Mapping process

7

Snomed-CTMedDRA

SOCHLPTHLTPT

ONTOADR.owl

• 55.6 % of MedDRA 13.0 terms could be defined using (i) a direct mapping with a Snomed-CT concept (UMLS or other mappings methods) or (ii) a handmade definition.

• Those terms cover 97.02 % of MedDRA terms used in the FDA database (calculated for the period 2004-2010: 11 millions reports).

Mapping process

Medical experts:

Validation of mappings

Manual enrichment of OntoADR

= MEDDRA TERMS + FORMAL DEFINITIONS

OntoADR generation: general schema

8

Thanks to OntoADR, OWL queries can be built to select the MedDRA PTs whose formal definition fits some definitional criteria.

Example : Query to catch MedDRA terms related to “Upper gastrointestinal bleeding”

Þ Will select from MedDRA hierarchy all PTs matching those two properties:

Duodenal ulcer haemorrhageGastric haemorrhageMallory-Weiss syndromePeptic ulcer haemorrhageetc.

hasAssociatedMorphology some ‘Hemorrhage’ AND hasFindingSite some ‘Upper gastrointestinal tract structure’

Using OntoADR to perform automatic query-based MedDRA terms groupings

9

Through the subsomption mechanism, MedDRA terms referring to hemorrhages of parts of the Upper gastrointestinal tract are also selected.

ex. Oesophageal ulcer haemorrhage, Gastric varices haemorrhage, etc.

Using OntoADR to perform automatic query-based MedDRA terms groupings

10

Focus on 13 ADR safety topics identified by Trifirò et al (2009) as first importance pharmacovigilance targets (EU-ADR project).

For each safety topic:

i. Groupings of MedDRA PTs have been realized with OntoADR queries.

ST 1 Bullous eruptionsST 2 Acute renal failureST 3 Anaphylactic shockST 4 RhabdomyolysisST 5 Aplastic anaemia/pancytopeniaST 6 NeutropeniaST 7 Cardiac valve fibrosisST 8 Extrapyramidal disordersST 9 Confusional stateST 10 ThrombocytopeniaST 11 Upper gastrointestinal bleedingST 12 Peripheral neuropathyST 13 Maculo-papular erythematous eruptions

ADR topics identified by Trifirò et al (2009)

Evaluation of the OntoADR-based grouping method

Trifirò G, Pariente A, Coloma PM, Kors JA, Polimeni G, Miremont-Salamé G, Catania MA, Salvo F, David A, Moore N, Caputi AP, Sturkenboom M, Molokhia M, Hippisley-Cox J, Acedo CD, van der Lei J, Fourrier-Reglat A. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf. 2009; 18(12):1176-84.

ii. The content of those groupings has been evaluated by comparison with existing handmade MedDRA groupings of PTs targeting same or close conditions:

A. Original MedDRA hierarchy groupings (HLTs or HLGTs)

B. SMQs (Standard Medical Queries): collections of MedDRA PTs developed manually by the MSS0 (Maintenance and Support Services Organization) to describe a common clinical condition

Neutropenia Safety Topic

Type (for SMQ)

Label Id Meddra

Agranulocytosis 10001507 Autoimmune neutropenia 10055128 Cyclic neutropenia 10053176 Febrile neutropenia 10016288 Felty's syndrome 10016386 Granulocytopenia 10018687 Granulocytopenia neonatal 10018688 Idiopathic neutropenia 10051645 Infantile genetic agranulocytosis 10052210 Neutropenia 10029354 Neutropenia neonatal 10029358 Neutropenic colitis 10062959 Neutropenic infection 10059482 Neutropenic sepsis 10049151

Narrow Agranulocytosis 10001507Narrow Band neutrophil count decreased 10057950Narrow Band neutrophil percentage decreased 10059130Narrow Cyclic neutropenia 10053176Narrow Febrile neutropenia 10016288Narrow Idiopathic neutropenia 10051645Narrow Neutropenia 10029354Narrow Neutropenic infection 10059482Narrow Neutropenic sepsis 10049151Narrow Neutrophil count decreased 10029366Broad Myelocyte percentage decreased 10052227Broad Neutropenia neonatal 10029358Broad Neutrophil count abnormal 10061313Broad Neutrophil percentage decreased 10052223

Leukopenia Sometimes used as synonym of Neutropenia

But stricly speaking Leukopenia semantically broader (refers to a deficit in the number of all types of white blood cells)

To enable comparison with query-based neutropenia grouping, only neutropenia relevant PTs were selected in the SMQ

HLT Neutropenias

SMQ Leukopenia

Generally defined as “an abnormally low number of neutrophils” (*), which are the most abundant type of white blood cells (leukocytes) in mammals.

(*) http://www.nlm.nih.gov/medlineplus/ency/article/007230.htm

hasDefinitionalManifestation some Neutropenia

OR interprets some (hasComponent some 'Segmented neutrophil' OR hasComponent some 'Myelocyte' OR hasComponent some 'Stab form')

AND (hasInterpretation some 'Below reference range' OR hasInterpretation some 'Decreased' OR hasInterpretation some 'Abnormal')

Neutropenia: Building the OWL query

Semantic relations used by the query

Description

interprets Refers to the entity being evaluated or interpreted, when an evaluation, interpretation or “judgment” is intrinsic to the meaning of a concept.

hasInterpretation This attribute is grouped with the attribute “Interprets”, and designates the judgment aspect being evaluated or interpreted for a concept (e.g., presence, absence, degree, normality, abnormality, etc.).

hasComponent Refers to what is being observed or measured by a procedure.hasDefinitionalManifestation

Links disorders to the manifestations (observations) that define them.(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte






Description




Links disorders to the manifestations (observations) that define them.(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte

Selecting disorders defined by an abnormally low number of neutrophils (ex. autoimmune neutropenia, cyclical neutropenia, etc.)






Description




Links disorders to the manifestations (observations) that define them.

Neutrophils or neutrophil granulocytes “are generally referred to as either neutrophils or polymorphonuclear neutrophils (or PMNs), and are subdivided into segmented neutrophils (or segs) and banded neutrophils [also called band neutrophil or stab cell]. They form part of the polymorphonuclear cell family (PMNs) together with basophils and eosinophils.” (*)

(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte

Myelocyte = precursor cell in the development of the granulocyte series.

Selecting analyses measuring quantity of neutrophils cells or neutrophils precursor cells(ex. Myelocyte count, Myelocyte percentage)






Description




Links disorders to the manifestations (observations) that define them.

Neutrophils or neutrophil granulocytes “are generally referred to as either neutrophils or polymorphonuclear neutrophils (or PMNs), and are subdivided into segmented neutrophils (or segs) and banded neutrophils [also called band neutrophil or stab cell]. They form part of the polymorphonuclear cell family (PMNs) together with basophils and eosinophils.” (*)

(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte

Myelocyte is precursor cell in the development of the granulocyte series.

Selecting abnormal / low results of those analyses(ex. Neutrophil count abnormal, Myelocyte percentage decreased, etc.)

Results: comparison with HLTType (for

SMQ)Label Id Meddra Q Label Id Meddra

Agranulocytosis 10001507 Y Agranulocytosis 10001507 Autoimmune neutropenia 10055128 Y Autoimmune neutropenia 10055128 Cyclic neutropenia 10053176 Y Band neutrophil count decreased 10057950 Febrile neutropenia 10016288 Y Band neutrophil percentage decreased 10059130 Felty's syndrome 10016386 Y Cyclic neutropenia 10053176 Granulocytopenia 10018687 Y Febrile neutropenia 10016288 Granulocytopenia neonatal 10018688 Y Felty's syndrome 10016386 Idiopathic neutropenia 10051645 Y Granulocytopenia 10018687 Infantile genetic agranulocytosis 10052210 - Granulocytopenia neonatal 10018688 Neutropenia 10029354 Y Idiopathic neutropenia 10051645 Neutropenia neonatal 10029358 Y Myelocyte count decreased 10050986 Neutropenic colitis 10062959 Y Myelocyte percentage decreased 10052227 Neutropenic infection 10059482 Y Neutropenia 10029354 Neutropenic sepsis 10049151 Y Neutropenia neonatal 10029358

13 Neutropenic colitis 10062959Narrow Agranulocytosis 10001507 Y Neutropenic infection 10059482Narrow Band neutrophil count decreased 10057950 Y Neutropenic sepsis 10049151Narrow Band neutrophil percentage decreased 10059130 Y Neutrophil count abnormal 10061313Narrow Cyclic neutropenia 10053176 Y Neutrophil count decreased 10029366Narrow Febrile neutropenia 10016288 Y Neutrophil percentage abnormal 10058134Narrow Idiopathic neutropenia 10051645 Y Neutrophil percentage decreased 10052223Narrow Neutropenia 10029354 Y Neutrophil percentage decreased 10052223Narrow Neutropenic infection 10059482 YNarrow Neutropenic sepsis 10049151 YNarrow Neutrophil count decreased 10029366 YBroad Myelocyte percentage decreased 10052227 YBroad Neutropenia neonatal 10029358 YBroad Neutrophil count abnormal 10061313 YBroad Neutrophil percentage decreased 10052223 Y

14

HLT

SMQ

Recall 100,0%Precision 59,1%F-measure 74,3%

100,0%

63,6% 77,8%

HLT

SMQ

OntoADR OWL Query

Results: comparison with HLTType (for

SMQ)Label Id Meddra Q Label Id Meddra

Agranulocytosis 10001507 Y Agranulocytosis 10001507 Autoimmune neutropenia 10055128 Y Autoimmune neutropenia 10055128 Cyclic neutropenia 10053176 Y Band neutrophil count decreased 10057950 Febrile neutropenia 10016288 Y Band neutrophil percentage decreased 10059130 Felty's syndrome 10016386 Y Cyclic neutropenia 10053176 Granulocytopenia 10018687 Y Febrile neutropenia 10016288 Granulocytopenia neonatal 10018688 Y Felty's syndrome 10016386 Idiopathic neutropenia 10051645 Y Granulocytopenia 10018687 Infantile genetic agranulocytosis 10052210 - Granulocytopenia neonatal 10018688 Neutropenia 10029354 Y Idiopathic neutropenia 10051645 Neutropenia neonatal 10029358 Y Myelocyte count decreased 10050986 Neutropenic colitis 10062959 Y Myelocyte percentage decreased 10052227 Neutropenic infection 10059482 Y Neutropenia 10029354 Neutropenic sepsis 10049151 Y Neutropenia neonatal 10029358

13 Neutropenic colitis 10062959Narrow Agranulocytosis 10001507 Y Neutropenic infection 10059482Narrow Band neutrophil count decreased 10057950 Y Neutropenic sepsis 10049151Narrow Band neutrophil percentage decreased 10059130 Y Neutrophil count abnormal 10061313Narrow Cyclic neutropenia 10053176 Y Neutrophil count decreased 10029366Narrow Febrile neutropenia 10016288 Y Neutrophil percentage abnormal 10058134Narrow Idiopathic neutropenia 10051645 Y Neutrophil percentage decreased 10052223Narrow Neutropenia 10029354 Y Neutrophil percentage decreased 10052223Narrow Neutropenic infection 10059482 YNarrow Neutropenic sepsis 10049151 YNarrow Neutrophil count decreased 10029366 YBroad Myelocyte percentage decreased 10052227 YBroad Neutropenia neonatal 10029358 YBroad Neutrophil count abnormal 10061313 YBroad Neutrophil percentage decreased 10052223 Y

14

HLT

SMQ


100,0%

63,6% 77,8%

HLT

SMQ

OntoADR OWL Query

Perfect recall and correct precision

Þ All PTs present in the handmade reference groupings can be selected automatically

Label Id Meddra Q Label Id Meddra

Acquired epidermolysis bullosa 10056508 Y Acne pustular 10000513Application site vesicles 10048941 Y Acquired epidermolysis bullosa 10056508Benign familial pemphigus 10004265 - Acute generalised exanthematous pustulosis 10048799Blister 10005191 Y Application site pustules 10049044Blood blister 10005372 Y Application site vesicles 10048941Bullous impetigo 10006563 Y Blister 10005191Cervical bulla 10050019 Y Blister infected 10005192Coma blister 10069827 Y Blood blister 10005372Dermatitis bullous 10012441 Y Bovine pustular stomatitis virus infection 10006048Dermatitis herpetiformis 10012468 Y Bullous impetigo 10006563Diabetic bullosis 10062356 Y Cervical bulla 10050019Epidermolysis 10053177 N Coma blister 10069827Epidermolysis bullosa 10014989 Y Dermatitis bullous 10012441Erythema multiforme 10015218 N Dermatitis herpetiformis 10012468Herpes gestationis 10019939 N Diabetic bullosis 10062356Implant site vesicles 10063870 Y Eczema herpeticum 10014197Incision site blister 10067990 Y Eosinophilic pustular folliculitis 10052834Infusion site vesicles 10065491 Y Eosinophilic pustulosis 10049172Injection site vesicles 10022111 Y Epidermolysis bullosa 10014989Linear IgA disease 10024515 N Heat rash 10019343Lip blister 10049307 Y Herpes zoster oticus 10063491Ocular pemphigoid 10067776 Y Impetigo herpetiformis 10021534Oculomucocutaneous syndrome 10030081 Y Implant site pustules 10063866Paraneoplastic pemphigus 10057056 Y Implant site vesicles 10063870Pemphigoid 10034277 Y Incision site blister 10067990Pemphigus 10034280 Y Infusion site pustule 10065487Penile blister 10052898 Y Infusion site vesicles 10065491Porphyria non-acute 10036186 N Injection site pustule 10054994Pseudoporphyria 10037145 N Injection site vesicles 10022111Staphylococcal scalded skin syndrome 10041929 N Lip blister 10049307Stevens-Johnson syndrome 10042033 N Ocular pemphigoid 10067776Toxic epidermal necrolysis 10044223 N Oculomucocutaneous syndrome 10030081Vaccination site vesicles 10069623 Y Paraneoplastic pemphigus 10057056 TOTAL 23

Pemphigoid 10034277Pemphigus 10034280Penile blister 10052898Pustular psoriasis 10037575Rash follicular 10037857Rash pustular 10037888SAPHO syndrome 10051316Subcorneal pustular dermatosis 10042342Urticaria vesiculosa 10046755Vaccination site pustule 10066047Vaccination site vesicles 10069623


Bullous eruptions Safety Topic

HLT Bullous conditions OntoADR OWL Query

This is not always the case!For other safety topics some PTs can’t be caught with the query method

19

Overview of results

Recall, precision and F-measure rates of automatic OntoADR groupings. SELECT (third column) indicates that a manual selection of PTs has been made in the MedDRA grouping taken as gold standard. (++) MedDRA reference groupings having a perfect semantic match with the safety topic. (+) MedDRA reference groupings having an imperfect semantic match with the safety topic (broader or narrower topic).

20

Overview of results


Our query-based grouping method allows catching most of the terms present in the MedDRA reference groupings.

21

Overview of results


Better recall and precision rates for HLT reference groupings than for SMQ

22

Overview of results


Very good recall and correct precision when considering only Meddra groupings perfectly matching the Safety topics

23

Results demonstrate that OntoADR and this OWL query-based grouping method can efficiently support the realization of automatic MedDRA terms groupings.

Promising result: MedDRA terms groupings for pharmacovigilance (mainly SMQs) so far achieved manually An automation of the process could allow an important saving of time

Next step: (a) Some terms of the handmade reference MedDRA groupings not retrieved by our automatic queries, and (b) new terms are caught.Does this difference in content has an impact (and if so is it positive?) on the signals detected by traditional statistical methods?

Þ Further study to address the question of the reliability of those query-based groupings for signal detection (comparison with standard MedDRA groupings - first of all of SMQs - signal detection performances).

Forthcoming: Souvignet, J., Declerck, G., Trombert, B., Rodrigues, J.M., Jaulent, M.C., Bousquet, C. Evaluation of automated term groupings for detecting anaphylactic shock signals with drugs. AMIA 2012, American Medical Informatics Association Annual Symposium, 3-7 novembre 2012, Chicago (US).

Conclusion

24

ANNEX. Snomed-CT relations used in OntoADR

25

ANNEX. Snomed-CT relations used in OntoADR

26

Only about 50% of MedDRA terms are mapped with Snomed-CT concepts within UMLS. A complete mapping of MedDRA seems very difficult to realize. Especially most MedDRA terms have no direct equivalent in Snomed-CT: their meaning must be decomposed to be mapped.

Ex.: MedDRA « Joint dislocation postoperative » |10049912| : No direct Snomed-CT equivalentÞ Decompositon of MedDRA term to perform the mapping:

• Snomed-CT « Dislocation of joint » |108367008| • Snomed-CT « Postoperative complication » |385486001|

A single MedDRA term can be mapped to several Snomed-CT concepts within the same UMLS concept. Þ UMLS embed one-to-one mappings but also one-to-several mappings.Þ Necessity to make manual selection to get a unique definition.

ANNEX. UMLS mapping difficulties

27

UMLS synonymy links between MedDRA terms and Snomed-CT concepts are not always correct.

Sometimes one term is more specific than the other.Ex. MedDRA ‘Pericardial neoplasm’ |10054945| is mapped with Snomed-CT ‘Neoplasm of uncertain behavior of pericardium’ |94997003| and ‘Neoplasm of pericardium’ |126734005|

Ex. MedDRA ‘Spondylitis’ |10061371| is mapped with Snomed-CT ‘Undifferentiated spondylitis’ |399096009|, ‘Inflammatory spondylopathy’ |202649003| and ‘Spondylitis’ |84172003|

Sometimes terms are not at all synonyms: Ex. MedDRA ‘Vascular disorders’ |10047065| refers to disorders of blood and lymphatic systems.Snomed-CT ‘Vascular disorder’ |27550009| only refers to disorders of blood vessels (‘Disorder of lymphatic system’ is used to refer to lymphatic system troubles)

With our mapping method, 39.8 % of MedDRA 13.0 terms (i.e. 8305 terms) were mapped via a one-to-one mapping and 9.5 % (i.e. 1984) were mapped via a one-to-several mapping.

ANNEX. UMLS mapping difficulties