WG-23 Status Report – 29 October 2007. Targets for WG-6 Overview for WG-6 – today Review for...

WG-23 Status Report – 29 October 2007

Targets for WG-6

Overview for WG-6 – today Review for Public Comment in March

2008 Frozen Draft in May/June 2008

May decide to ballot early if implementations are far enough along

Ballot Fall 2008

Basic Goals of Supp. 118

Simple encapsulation of existing applications Self-contained applications Data exchange via files (e.g. Part 10 format)

Ease development of new applications for DICOM novices Simple abstract model Hosting System parses and builds DICOM

objects Full native access available if needed

Use existing technology where practical

Out of Scope for Supp. 118 Full integration into the Hosting System GUI

No standard GUI widgets However, host may provide rectangular screen

area recommendation to cooperating applications

Discovery No standard service for locating suitable

applications on the Internet for installation No mechanism for determining what applications

are installed on the Hosting System Access to DICOM services beyond storage

In Debate

Formalized description of applications Against – not enough time to ‘get it right’ For – needed to verify application ID (e.g. signature) Alternative – a paper-based description

Start-up method – command line arguments Against – does not cover all start-up methods For – simple, available on most systems Alternative 1 – well-known port Alternative 2 – args, with well-known port fallback Alternative 3 – leave it open, let host decide

Application Initialization1. Hosting System locates desired Application

using mechanisms outside of the standard.2. Hosting System initializes the Hosted

Application using the equivalent of a “run” or “exec” command.

3. Hosted Application utilizes command line arguments passed by the Hosting System to connect to the “Host” interface, and to present the “Application” interface.

4. If no command line arguments, the Hosted Application utilizes a “well known port” to obtain the needed information for connecting to the “Host” interface, and to present the “Application” interface.

Two Main Interfaces

“Application” holds methods that the Hosting System uses to control and feed data to the Hosted Application.

“Host” holds methods that the Hosted Application uses to communicate outputs to the Hosting System, and to notify it of status and state changes.

One-to-one correspondence of instances of the “Host” and “Application” interfaces.

Interfaces are defined in WSDL to be language and platform independent.

“Application” Methods

setState(state : State) : boolean getState() : State bringToFront() : boolean

“Host” Methods

notifyStateChanged(state : State) : void notifyOutputAvailable() : void notifyStatus(status : Status) : void getTmpDir() : String getOutputDir() : String generateUID() : String getAvailableScreen

(appPreferredScreen : Rectangle) : Rectangle

Application States

IDLE – waiting for next task INPROGRESS – performing task COMPLETED – task done, wait for

Hosting System to grab output SUSPENDED – task is not finished,

but no processing is being done CANCELED – kill the task and release

resources with no further output EXIT – destroy this instance of the

application

Application Running

On start-up, the Hosted Application waits in the “IDLE” state.

The Hosting System triggers a task by changing the Hosted Application’s state to “INPROGRESS”.

Hosted Application grabs the input data. The Hosted Application only handles

one task at a time. The Hosting System may

simultaneously start multiple Hosted Applications, including more than one running instance of the same Hosted Application.

Application Suspend

The Hosting System may pause a running Hosted Application by changing the state from “INPROGRESS” to “SUSPENDED”.

When suspended, the Hosted Application minimizes resource use without loosing task context.

The Hosting System asks the Hosted Application to continue processing the task by changing the state to “INPROGRESS”.

Application Cancel

The Hosting System kills a running or suspended task by changing the Hosted Application’s state to “CANCELED”.

The Hosted Application aborts processing, releases all resources, and returns to the “IDLE” state, waiting for the next task.

Application Task Completion When the Hosted Application has

completed its task, it changes its state to “COMPLETED” and notifies the Hosting System of the state change.

When the Host System has collected the output data, the Hosted Application changes state to “IDLE”, and notifies the Hosting System of the state change.

Application Termination

If the Hosted Application is in the “IDLE” state, the Hosting System may terminate the Hosted Application by changing the state to “EXIT”.

Condition Handling

If there is a notable condition (e.g. a progress report, an error) in the Hosted Application, it may inform the Hosting System via a notifyStatus() call.

The Status includes: statusType (i.e. INFO, WARN, ERROR,

FATAL) codingSchemeDesignator codeValue messageString

Data Exchange

File Access Methods Simple exchange of files (e.g. DICOM Part 10) Can support other file formats (e.g. Analyze)

used by researchers Furthest along (essentially finished)

Model Access Methods Both Native (e.g. DICOM models) and Abstract

(e.g. Multi-Dimensional Image) versions Uses commonly available tools that are often

used to process XML Is independent of the underlying data format

File Access Methods

getNativeObjectDescriptors() : NativeObjectDescriptor[]

getAsFile(desireObjects : NativeObjectDescriptor[]) : NativeObjectLocator[]

Native Object Descriptor

Consists of: UUID : String MIMEType : String SOPClassUID : String

Used to describe the native form of one object (e.g. a DICOM Part 10 file)

UUID used to: Avoid potential collisions with SOP Instance

UID Maintain generality

Native Object Locator

Consists of: referencedUUID : String uri : URI

referencedUUID, which identifies the object, is taken from the Native Object Descriptor

The uri identifies the location of the desired object (i.e., the file)

File Exchange Sequence

Recipient Sender

getNativeDescriptors()

return of NativeDescriptors[]

getAsFile( targets NativeDescrptors)

return of NativeLocators[]

Symmetric File Exchange

File Exchange methods are symmetric (i.e. the Hosted Application uses the same methods to get input data from the Hosting System that the Hosting System uses to get output data from the Hosted Applications)

Once the recipient asks for a Native Object Descriptor as a file, and the sender responds, then the sender takes the object off of the Native Object Descriptor list

Model Access Methods

Derived from Java Image IO concepts: Abstract access to common data Generic mechanism to access format-specific

data Utilizes the “XML Infoset” concept

Hosting System maintains a model of the referenced data, e.g. using DOM tools Using DOM does not mean that the data ever

existed natively in XML form DOM is a convenient way to describe the layout of

the data, even if the data is in DICOM format Hosted Application utilizes XPath to identify the

desired data

Example XPath for Native Model/DicomObject/ViewCodeSequence/

Item[@number=1]/ CodeMeaning/value[@number=1]

Will add a column to Part 6 Data Dictionary for properly formatted tag IDs

Will have provisions for proper Private Data Element access

Can access by Group and Element Tags instead of by tag ID

Alternative Without XPathgetNativeMetadata()

.getElementsByTagName(“DicomObject”).item(0) ()

.getElementsByTagName(“ViewCodeSequence”).item(0)

.getElementsByTagName(“Item”).item(0)

.getElementsByTagName(“CodeMeaning”).item(0)

.getElementsByTagName(“value”).item(0)

.getNodeValue()

Proposed Model Methods

Get Abstract Object Descriptors Alternative to getNativeObjectDescriptors() Hosting System creates Abstract Models as needed Returns Native Descriptors if not part of an

Abstract Model Get/Set XPath

May return single values or Infosets Set may be restricted to values

Asymmetric Hosting System manages the creation of models Create new models related to old ones

Allows Hosting System to track info needed to serialize the object in DICOM

Allows Hosting System to add derived references

Abstract Model (in development) Make life easier for the application developer Draws simplified concepts from the new DICOM

enhance multi-frame objects Only commonly used dimensions included References the source native models if an

application needs full details Assumes ‘cooked’ data, e.g.

Modality LUT applied ‘Clean’ data (sign extended, no overlay bits) Data from old style SOP Classes reorganized Pixel interleaved color only

Signed & unsigned integers, single & double floats

Semantic intent and units included

Access to Bulk Data

“Frame” is the smallest common denominator

Bulk Data References done as file locators plus offsets to start of data Can be read in with normal I/O, or Can be accessed as memory-mapped

files

Implementation Status

Three independent implementations of earlier version as ‘proof of concept’, including Java and .net versions interoperating

Primitive performance benchmarking Two independent implementations of

the file-based proposal Methodology similar to the model access

methods used in other projects Memory-mapped access used in other

projects

Driving Forces

The eXtensible Imaging Platform™ project from NCI’s caBIG™ program First open source release due 15 December

2007 ‘Hands on’ demonstration at RSNA 2007 ‘Hands on’ tutorials at SIIM 2008 Initial focus on analysis applications for trials

Two hour workshop at Medical Imaging 2008

Multiple implementers

caBIG will facilitate sharing of infrastructure, applications, and data

Core Principles

• Common, widely distributed informatics platform

• Shared vocabulary, data elements, data models

• Common standard for developing applications

In Vivo Imaging Workspace

• Middleware• NCIA• geACRIN• AIM • XIP

The eXtensible Imaging Platform (XIP™) is the image analysis and visualization tool for caBIG.

XIP is an open source environment for rapidly developing medical imaging applications from an extensible set of modular elements.

XIP may be used by vendors to prototype or develop new applications.

Imaging applications developed by research groups will be accessible within the clinical operating environment, using a new DICOM Plug-in interface first implemented in XIP.

XIP may server as a reference implementation of the DICOM WG-23 Application Hosting interfaces.

What is the ?

XIP Application

(Can be replaced with any DICOM WG23-compatible Host)

XIP Host Adapter

XIP

LIB

ITK

VTK

. . .

XIP ModulesHost Independent

WG23

XIP HostWG23

WG23

Web-based Application

Medical Imaging Workstation

Standalone Application

Distribute

Distribute

DICOM, HL7, and otherservices per IHE Profiles

caGRID Services viaImaging Middleware

XIP Application Builder

XIP Class Library Auto Conversion Tool

Host-Specific Plug-in Libraries

WG23

Distribute

WG-23 API (Socket)WG-23 API (Socket)WG-23 API (Plug)WG-23 API (Plug)

XIP ApplicationXIP Application

Custom Custom XIP XIP

ClassesClasses

StandardStandardXIP LibraryXIP Library

ClassesClasses

IVI Middleware caGRID Interface

DICOM/IHEIntefaces

XIP Plug-in Architecture

XIP HostXIP Host

DICOM Plug-in Framework

• WG-23 addresses clinical integration and vendor inter-operability by defining standardized “plugs” and “sockets” (APIs)•caBIG XIP addresses an open-architecture, open-source integrated development environment for rapid prototyping & collaboration based on WG 23 APIs.

Unix, Mac, PC Internet ServerCommercial Vendor

#2

…

Commercial Vendor #1

Clinical Prototype & Collaboration

XIP developed

Application

Standard API

After Supp. 118?

Work item proposal for Phase 2 at spring 2008 DICOM Standards Committee meeting

Phase 2 will fill holes from Phase 1 (e.g. some of the ‘out of scope for 118’ issues)

Abstract metadata for multidimensional image data

considered as functions

B Gibaud

25/10/2007

Goals

• Provide documentation of the bulk data• Concerning

– logical organization• Range of the function : scalar or not, data type used (uchar,

char, int, float, double, etc)• Domain of the function : Number of variables, order of

variables, number of samples along each dimension, regular sampling or not

– Semantics• Of components• Of variables

Spec of domain and range

• Range (components)– Scalar

• Data-type, e.g. int32• Semantics, e.g. T1weightedMRIsignalintensity

– Vector• Nb-components ; table of (Data-type, Semantics)

• Domain (variables)– Nature of interval

• Regular-interval– Nb-samples ; inter-sample-distance ; sample-width

• Non regular interval– Nb-samples ; origin ; table of (dist-to-origin, sample-width )

– Semantics : e.g. space, time, energy

Example 1

• T1weighted MR dataset– Scalar Range

• Data-type=int32• Semantics = ‘T1weightedMRsignalintensity’

– 3 variables• Regular interval

– Nb-samples=256, inter_sample-dist=1mm, sample-width=1mm– semantics = ‘space along X’

• Regular interval– Nb-samples=256, inter_sample-dist=1mm, sample-width=1mm– semantics = ‘space along Y’

• Regular interval– Nb-samples=120, inter_sample-dist= 4mm, sample-width=4mm– semantics = ‘space along Z’

Example 2• fMRI dataset

– Scalar Range• Data-type=int16• Semantics = ‘T2STARMRsignalintensity’

– 4 variables• Regular interval

– Nb-samples=128, inter_sample-dist=4mm, sample-width=4mm– semantics = ‘space along X’

• Regular interval– Nb-samples=128, inter_sample-dist=4mm, sample-width=4mm– semantics = ‘space along Y’

• Regular interval– Nb-samples=12, inter_sample-dist= 9mm, sample-width=9mm– semantics = ‘space along Z’

• Regular interval– Nb-samples=200, inter_sample-dist= 1s, sample-width=0.5s– semantics = ‘time’

Example 3

• SPECT acquisition dataset (TOMO)– Scalar Range

• Data-type=int16• Semantics = ‘Number of counts’

– 3 variables• Regular interval

– Nb-samples=128, inter_sample-dist=4mm, sample-width=4mm– semantics = ‘space along X’

• Regular interval– Nb-samples=128, inter_sample-dist=4mm, sample-width=4mm– semantics = ‘space along Y’

• Regular interval– Nb-samples=128, inter_sample-dist= 2.81°, sample-width=2.81°– semantics = ‘space along theta (projection angle)’

Example 4

• 3D displacement field (non linear registration)– Vector Range

• 3 components– (Data-type=float ; Semantics = ‘space displacement along X’)– (Data-type=float ; Semantics = ‘space displacement along Y’)– (Data-type=float ; Semantics = ‘space displacement along Z’)

– 3 variables• Regular interval

– Nb-samples=256, inter_sample-dist=1mm, sample-width=1mm– semantics = ‘space along X’

• Regular interval– Nb-samples=256, inter_sample-dist=1mm, sample-width=1mm– semantics = ‘space along Y’

• Regular interval– Nb-samples=120, inter_sample-dist= 4mm, sample-width=4mm– semantics = ‘space along Z’

Example 5-1

• RGB 2D image – Vector Range

• 3 components– (Data-type=int16 ; Semantics = ‘Luminance in Red’)– (Data-type=int16 ; Semantics = ‘Luminance in Green’)– (Data-type=int16 ; Semantics = ‘Luminance in Blue’)

– 2 variables• Regular interval

– Nb-samples=1024, inter_sample-dist=0.5mm, sample-width=0.5mm

– semantics = ‘space along X’• Regular interval

– Nb-samples=1024, inter_sample-dist=0.5mm, sample-width=0.5mm

– semantics = ‘space along Y’

Open issues 1

• Need to have a sort of ‘qualitative variable’ to manage e.g. RGB images in 3 separate planes, indexed by this variable ?– semantics of the corresponding variable would be :

‘Red’, ‘Green’, ‘Blue’ ?– semantics of the corresponding (scalar) range would

be ‘luminance’ ?

• Probably needed• It would become somewhat arbitrary to choose

between a « vector range » versus a colour qualitative variable

Example 5-2

• RGB 2D image (as 3 separate planes)– Scalar Range

• (Data-type=int16 ; Semantics = ‘Luminance’)

– 3 variables• Regular interval

– Nb-samples=1024, inter_sample-dist=0.5mm, sample-width=0.5mm– semantics = ‘space along X’

• Regular interval– Nb-samples=1024, inter_sample-dist=0.5mm, sample-width=0.5mm– semantics = ‘space along Y’

• Qualitative variable– Nb-samples=3– (Semantics = ‘Luminance in Red’)– (Semantics = ‘Luminance in Green’)– (Semantics = ‘Luminance in Blue’)

Open issues 2

• Need to manage ‘related variables’ ? for instance in the SPECT example (TOMO), the indexing could be done both on the ‘projection angle’ and on ‘time’– Useful ?– Is sampling necessarily regular with both

variables (linear relationship between the two) ?

Open issues 3

• Need to manage units in which scalar components are represented (Hounsfield, NM units, etc.) ?– Useful ?

Example 6

• MR Spectro– TBD