Workshop: Reference Image Datasets for Medical Image ... Workshop: Reference Image Datasets for Medical…

Embed Size (px)

Text of Workshop: Reference Image Datasets for Medical Image ... Workshop: Reference Image Datasets for...

  • 1

    Workshop: Reference Image Datasets for Medical Image Processing Research and Development

    August 27, 2002, 16:10-17:30

    Alexander HorschChair of WG MIP

    Topic summary MIP is of increasing importance for modern medicine

    (CAD, CAS, etc) However: Results of R&D work are usually not or hardly

    comparable due to the fact that each R&D group uses its own datasets for to develop and evaluate their methods

    Furthermore: datasets themselves are often insufficient or too small for clinically sound evaluation studies

    There is an urgent need to establish, step by step, a reference image database providing numerous high-quality datasets for medical image processing R&D groups from different domains.

    In September 2001, with the EFMI WG an initiative has been started to begin this work.

  • 2

    Goal of the workshop During the workshop, the existing concept and roadmap for to

    create such a database, as well as first available datasets shall be presented and discussed.

    In the end of the workshop, a list of recommendations for a revision of both concept and roadmap shall be agreed on.

    Furthermore, interested colleagues can join the efforts of EFMI WG MIP.

    Professional interests of audience

    intended audience are colleagues working in the field of medical image processing, either on methodological research or on development of clinical applications like CAD or CAS. Possible motivations for to join the workshop can be: to get information about existing material and how to get it

    for the own work identification of possible own contributions to the database for to get up-to-date information about the subject for to meet experts in this field

  • 3

    Expected outcome A list of recommendations on further development

    of reference database New contacts between the WG and potential

    participants in the development

    Agenda Presentation of

    Background Concept Roadmap First Datasets

    Presentation of Casim@ge (Christian Lovis, Geneva) Discussion of the presented material Collection (brainstorming) and sorting of ideas for

    improvement Selection and final agreement on a list of


  • 4


    Image -recordingImage -



    processing SegmentationSegmentationParameter-extraction

    Parameter-extraction ClassificationClassification

    ResultsResultsSceneScene ImageImage ImageImage ObjectsObjects FeaturesFeatures

    MIP Teaching

    Calibration Modality Simulators Standardized

    - acquisition- documentation of

    acquisition chains- paradigm / instruction- image format

    Image improvement Image Registration Image econstruction Image/Data Fusion 3D Volume

    (Re)construction Visualization Detection of ROIs Mapping VR Augmented Reality Image reconstruction from partial cquisition


    Object detection & counting Quantification/ Measurement Identification of anatomic structures Visualization Reference Validation

    CAD Pattern recognition Detection of location Fibre tracking from diffusion MRI

    Storage & documentation Image communication (telemedicine)

    CBIR Compression (lossy / non-lossy) Cryptography / Watermarking

    The MIP Specialty

    Background MIP is growing interdisciplinary field Basic research, clinical evaluation, algorithm design, clinical studies Own image datasets by almost each R&D group Algorithms tend to be optimized to the one or few acquisition units

    available Comparison of research results from different R&D groups rarely

    possible Datasets usually do not represent the medical variety needed for large

    clinical studies Solution multi-center studies usually suffers from unbalanced usage of

    different imaging units

  • 5

    EFMI WG MIP Initiated 09/2001 London (Medinfo) Mission:

    Foster discussion and activities on theory and practice of MIP comprising: integration of decision support by MIP into clinical practice

    (including clinical evaluation, standardization, technology transfer) Work products:

    RID-MIP MIP R&D Groups IS WG Website

    WG Members Austria: Prinz, Vogl Finland: Sipil Germany: Horsch, Wittenberg, Schneider, Zahlmann Switzerland: Valle The Netherlands: Verdonck

    Interested colleagues from further EFMI countries: Spain: Gomez Vilda

    coming from universities, Siemens, hospitals.

  • 6

    Agenda Presentation of

    Background Concept Roadmap First Datasets

    Presentation of Casim@ge (Christian Lovis, Geneva) Discussion of the presented material Collection (brainstorming) and sorting of ideas for

    improvement Selection and final agreement on a list of


    Overall Approach RIDSs (well-defined, validated) from experienced providers RIDSs meet quality criteria defined by interdisciplinary expert groups

    (domain-specific) Bilateral agreements / contracts between provider & users of a RIDS EFMI WG MIP

    proposes tasks / challenges supports RIDSs specifications supports RIDSs dissemination organizes special sessions at conferences evaluates overall impact

  • 7

    Criteria for Task Assessment Relevance of medical problem in terms of

    Mortality & morbidity statistics DRG statistics (e.g. USA, Australia, Austria) What is funded by WHO, EC, Current science activities, Literature

    Role of imaging in the diagnostic / therapeutic process key role, important role, marginal role

    Potential for improvement of outcome by improvement of image reading / image usage / image analysis life years, cost savings

    MIP task paths to find solutions exists MIP challenge no real ideas present of how to find a solution

    Leading causes of death, USA 1999

    From EFMI-WG-MIP-2310-00.10 draft 2002-07-24 Fig. 1: Part of Table 1: Deaths, percent of total deaths, and death rates for the 10 leading causes of death in selected age groups, by race and sex: United States, 1999. National Vital Statistics Reports, Vol. 49, Nr. 11, October 12, 2001, page 14

  • 8

    Medline Meta-Studies

    TumorstadieneinteilungLymphatische Metastasierung4



















    From EFMI-WG-MIP-2310-00.10 draft 2002-07-24 Table 1: Total number of meta-studies on diagnostic imaging referenced in Medline database on June 5, 2002, sorted by frequency

    Task/challenge proposals CAD for support of breast cancer screening:

    Detect micro-calcifications on all images of the mammography dataset D1 with a 10-fold cross-validated sensitivity > 0.9 at a rate of false positive ROI detections < 0.5! CHALLENGE

    CAD for esophagus tumor staging:Make a robust and fully automatic T1 to T3 staging of esophageal tumors in the EUS still images dataset D2 with an overall classification quality of > 0.6! CHALLENGE

  • 9

    Objectives of RID-MIP Provide a methodology for building a medical image database Provide information on existing datasets and who is the provider Provide templates for bilateral agreements / contracts between

    provider and users Provide different kinds of datasets for evaluation of MIP

    methods and applications Follow-up the usage of datasets in terms of published results Bring the prototypical RID-MIP into long-term operation (e.g. at a

    publishing company)

    Addressees of RID-MIP European MIP R&D groups

    MIP R&D groups outside Europe

    Public sector (usually free of charge)

    Private / commercial sector (usually by licensing)

  • 10

    MIP system development and the different kinds of datasets

    Modality Simulator Modality Simulator



    System TrialsSystem Trials

    Development time axis

    Clinical Multicenter Study

    Clinical Multicenter Study

    PatientInformationExamination Information

    --- (x)--- x x

    x (x)Parameters x x

    (x) x x xReference generated


    Licensing Policy ETDs method-oriented free (agreement) STDs research public sector free, agreement STDs product-oriented license fees user to provider,

    contract EFMI WG MIP will provide templates for agreements &


  • 11

    Provider/User Agreement 1 Subject

    ... timely limited, non-exclusive usage license for the dataset

    2 User license user is allowed to use the dataset in the framework of the

    following scientific studies:

    3 Binding to purpose 4 Validity timespan 5 Reference character of the dataset 6 Usage by a third party

    Provider/User Agreement 7 Media and copies 8 Reference copy 9 Fees 10 Co-operation 11 Publication of the results 12 Runtime and irregular termination

  • 12

    Organisation & Roles EFMI WG MIP (RID-MIP Project Group)

    initiating, supporting, organizing, assessing

    Technical Board of Experts specifying

    Medical Board of Experts recommending

    Dataset Provider preparing, submitting, o