Capturing and Modeling Neuro-Radiological

Knowledge on a Community

Basis: The Head Injury Scenario.

Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker,

and Suisheng Tang

Data Mining DepartmentInstitute for Infocomm Research

Singapore

Outline

• Motivation

• MiBank – Head Injury Database

• Ontology Development

• Collective Intelligence

• The “facebook” approach

• Medical Image Annotator

• Discussion and Conclusions

Motivation• National Neurological Institute Singapore (NII) has 500+

head injury patients each year with Brain, Scalp, Skull, Internal bleeding requiring rapid diagnosis.

• Clinical radiology reports comprise of multiple series of Computed Tomography (CT) Images with unstructured text associated to images)

• Computationally a weak association between images and words, cannot retrieve similar images.

• Conceptually a tightly coupled association between Image and Diagnosis

• MiBank database of DICOM files, (http://dicom.i2r.a-star.edu.sg/pacsone/)

Features of MiBank Browser

Search by category, patient, report, note, study

Annotation with free-text

Forum discussion

DICOM viewer

Image upload & download

MiBank: Medical Image Databank: Head Injury

Web site: http://dicom.i2r.a-star.edu.sg/pacsone/

505 studies, 1775 series, 31561 images. Pass word protect, DICOM viewer, searchable

Show me all cases who have skull fracture with acute subdural hematoma But do not have brain edema.

Possible query in MiBank

Show all cases who have skull fracture with midline shift and acute subdural hematoma But do not have brain edema.

Details in next page

Impossible query with no-

predefined terms

Current Limitations of MiBank

A fracture of the right frontal bone.

Mild midline shift to the left is present.

An acute extradural hematoma, measuring

1.9 cm in maximal thickness, is noted.

A 1 cm thick acute subdural hematoma is

also present over the right cerebral

hemisphere.

Sample: Radiology Report

Can not query based on image features explicitly;

Can not associate the description in R-report to specific instance of an image.

Need to see all instances for bone fracture

What do we want… What do we need?

• Retrieve patients with right midnight shifts of less than 3mm for whom there has been no reported haematoma

• Retrieve all images similar to this one

• Properly annotated data: images, radiology reports

• Meaningful associations between reports, images, and across images

• …an ontology ….

Head Injurydatabase

(Relational)

Categorization

Indexing

Header InfoMining

OriginalDICOM

data

Web In

terface

Search Engine

Visualization

Reportdata

Text mining

Image retrieve

Statistic report

Customized online report

Discussion forum

Semantic Query

Ontology

The Role of the Ontology

• Community defined controlled vocabulary for annotation of radiology images.

• Hierarchical descriptions of medical terms relevant to anatomy, pathology and head injury specific features found in medical images.

• Consensus model of head injury terminology generated through community engagement for knowledge reuse in medical information systems.

• Query model for semantic search

Ontology Development

Garcia et al

Ontology Development

P h a s e 1 P h a s e 2

Text Processing / Baseline Ontology

“Plain scans were acquired. Note is made of the MRI dated 2/3/2004 and CT dated 18/2/2004.Evidence of previous left high parietal craniectomy noted. Hypodensity in the left parietal-occipital region is compatible with gliosis at site of previous surgery. A large left-sided scalp hematoma is seen. Underlying linear radiolucency in the left frontal bone was seen. This suggests an undisplaced fracture. Underlying acute subdural hematoma is seen with a maximal depth of 1.2 cm. Acute subarachnoid blood is also noted collecting mainly in the ipsilateral cerebral hemisphere, sylvian fissure as well as tentorium. There is diffuse cerebral edema. Mass-effect is seen with midline shift to the right, and developing hydrocephalus. Basal cisterns are effaced”.

FMAFMA Non FMANon FMA

FMA / Galen / R-report terms: FMA / Galen / R-report terms: anatomy, pathology, trauma, injury anatomy, pathology, trauma, injury

Capturing Knowledge: Phase 1

• Requires excessive amount of time • Experts – easily bored – no short term

result. • Results in the creation of unstructured

knowledge stores that are difficult to reuse and maintain.

• Skimping on validation may include errors, omissions, inconsistencies & irrelevances

• Experts are not always capturing the evidence – rather explaining context

• Storing the knowledge that is not machine-readable

Not an easy task

• Inside expert’s head

• Difficult to describe

concepts and relations

• Difficult for non-

experts to understand.

Disadvantages

Ontology Development

P h a s e 1 P h a s e 2

Maintenance

Evolution

Capturing Knowledge: Phase 2

• Collective Knowledge Resources– intelligent collection?

• collaborative bookmarking, searching

– “database of intentions”• clicking, rating, tagging,

buying - Amazon

– what we all know but hadn’t got around to saying in public before

• blogs, wikis,

discussion lists -

• Knowledge Elicitation via Collective Intelligence– The capacity to provide

useful information based on human contributions which gets better as more people participate.

– Data Types• mix of structured,

machine-readable data and unstructured data from human input

Retrieving images of the diving trip to Australia. Albert and Alex have to be in the photo.

Tags Make The Difference !

• The Premise: From unstructured and unrelated annotation to structured meaningful annotation

• Simple tagging it possible to derive meaningful associations

• Need to have a tool to gather knowledge that is directly linked to supporting evidence.

facebook

Medical Image Annotator: MIA• Main challenge in medical image retrieval

is that it heavily depends on expert’s knowledge of data structures and annotation is poor. So the objective of MIA is knowledge capture.

• MIA is designed for medical image annotation and its users are domain experts who require a consistent vocabulary for annotation tasks, knowledge sharing and machine automation.

• User community consists of Radiologists, Neurosurgeons (specifically, NNI doctors). Medical students, junior doctors, image processing researchers.

• MIA is a designed to both facilitate the building of appropriate ontology by domain experts and effective maintenance and evolution of the ontology, given new use cases /images.

MIA User Interface

Our contribution: the use of WEB 2.0 technology to support knowledge capture, and the approach to community engagement in the development of the ontology; more concretely in the maintenance and evolution

MIA: Platform Architecture

Server-side processors Client-side browser

Image

Database

OntologyViewer

.owlfile

Java script (DHTML)

AJAX

Owl Parser

Tree Constructor

OntologyEditor

Ontology & ImageManagement Console

Ontologies can be edited online: * add node * rename node * delete node

OWL files can be loaded dynamicallyOWL relational database OWL

• Users can keep their own version of ontology• Consolidated ontology will be generated based on community inputs.

Ajax to update ontologies on server side to provide dynamic content on a web page so no page-refresh, no re-loading

Easy to extend, any OWL file can be loaded

Knowledge Capture in Action

Knowledge Capture in Action

Knowledge Capture in Action

Medical Image Annotator: MIAAdvantages

• Fast and easy• Domain experts lead the

process• Always rooted in reality or

a medical use case • Maintenance and evolution

of the controlled vocabulary is assured.

• Excellent training for new doctors / radiologists

• Facilitates Data Mining of Radiology reports

Ontology Evolution

• Different trainee and clinical doctors building ontologies with extensions on

different sub trees

• Consolidated ontology is currently manually curated

• Goal is automatically align & merge ontologies

Query with the Head Injury Ontology

1. Simple ‘ontology-term’ assisted query• Search for images: based merely on simple

combination of ontology terms (and / or)• Form based interface linked to SQL Queires

2. Ontology reasoning (A-box)• Content navigation over R-reports using defined

object properties (Knowlegtor)• Use of subsumption and object properties

Concepts

(27)

Roles

(38)

• BrainRegion • has_BrainRegion

Information

• Symptom • has_Symptoms

• DiseaseDiagnosis • has_DiseaseDiagnosis

• DiseaseStage • has_DiseaseStage

• ImageReport • has_ImageReport

• ImageView

• TextReport

• has_ImageView

• has_TextReport

• Intracranial

Hemorrhage

• has_Intracranial

Haemorrhage

• Intraventricular

Hemorrhage

• has_Intrventricular

Haemorrhage

Head Injury Ontology

Find patient records for ‘Fracture’

Discussion and Conclusions• Medical images should be better annotated in order to facilitate

information retrieval

• Collective knowledge is real… “FAQ-o-Sphere”

• Controlled vocabularies (CVs) and/or ontologies are being developed by communities

• Simple tagging combined with knowledge elicitation methods supports ontology development

• Collective knowledge capture requires dedicated infrastructure that supports specific tasks

• Querability can be improved through the use of explicit tags and CVs/ontologies

Social Web Social + Semantic Web

• How to get knowledge from all those intelligent people on the Internet

• How to give everyone the benefit of everyone else’s experience

• How to leverage and contribute to the ecosystem that has created today’s web.

Challenges for the Community

Life ScienceLife Science

Acknowledgments • Bonarges Aleman-Meza – Social Web• Tom Gruber - Semantic-Social Web• MIA Developers - Zhang Zhuo and Menaka

Rajapakse• Suisheng Tang M.D. and Project PI, - Coordinator

of domain experts and builder of baseline ontology• Tchoyoson Lim – Radiologist NNI (National

Neuroscience Institute, Singapore)