N-Dimensional Presentation States
Progress and Paths from WG 11
Michael Aratow, M.D., FACEPNicholas F. Polys Ph.D.
Web3D Consortium
Convergence in Medical ITNew opportunities in improving quality and
reducing cost of care:• Standards enable new analytics– Reproducible Interactive presentations and
reports– Leveraging the web multimedia ecology (W3C)– Semantics (FMA, SNOMED CT)
• Standards enable new levels of care– Consistent communication between providers and
organizations
Critical Path:Medical Imaging & Informatics
• Interoperability challenges for the presentation of medical images are well-recognized – Accessibility outside the radiology suite– Procedural and Cognitive Education– Informed Consent– Custom prostheses– Surgical Planning– Radiation Therapy and Planning
• Why risk a new wheel? – Proven and unencumbered scenegraph abstractions yield a
quantum jump: the ‘Hopscotch Strategy’
• A decade of experience with 3D and the web in an ISO environment– Aligning with proven web delivery and
presentation technologies
• Leads to the obvious reason these standards bodies are working together:– Existing DICOM requirements overlap existing X3D
functionality – ISO standardization – Open technology
The Hopscotch Strategy
The Web3D Consortium
A not-for-profit organization that develops and maintains ISO standards for the description and transmission of 4D data over telecommunication networks. The X3D, H-Anim and VRML standards specify both a file format and a runtime specification that are open and royalty-free
www.web3d.org
Web3D & X3DProven Innovation (see web3d.org):
• Real time interactive graphics – Lights, cameras, text, meshes, textures and materials– Animations, sensors, volumes & style
• Interoperability with web – distributed resources referred via URL/URI– E.G. images, sounds, videos, models, worlds
• Durability – ISO Specifications:
• Abstract Spec (scenegraph)• Encodings• Bindings• H-Anim
http://www.web3d.org/x3d/specifications/
Highlights
• Abstract scenegraph with multiple encodings and API bindings• Well-defined runtime semantics• ISO (SC-24), Royalty-Free• Multiple open-source and commercial implementations (approx 24)• demonstrated capability for medical applications• covers significant majority of DICOM requirements• remainder being specified through WG11 work item
US ARMY TATRC Contract
• Successfully completed in 2007• MedX3D Draft Specification available to
members• Implementations:– H3D.org – Open Source Haptics incl.
X3D, MedX3D and VRML– Fraunhofer IGD’s Instant Reality
• Semantic Scenegraph inline or referenced• Volume Rendering Demos
Web3D + DICOM Recent History
• Started in WG-17: Web3DC contributed to 3D Surface data structures (compatible with ISO 19775 (X3D))
• Now primarily in WG11: Scoped Work Item on n-Dimensional Presentation States
• Web3D Consortium Medical WG presenting specs and royalty-free technology meeting (and exceeding) requirements
Timeline Recap
DICOM: Critical Mass• Continue to gather momentum, requirements,
alignments with DICOM stakeholders (you!)• Web3D Consortium brings a time-proven
solution: a standardized scenegraph to meet the requirements of the healthcare enterprise
• Recognizing the value of consistent renderings across caregivers and stakeholders, the US Army TATRC provided critical support (spec draft and open source implementations)
We are pleased to report: Significant progress on this work item!
X3D Interactive Presentations use DNA
Basic Interactive Scenes:• 2D & 3D Geometry, Appearance, Text, Layers• Lighting• Sensors• Event Utilities• Interpolators, Sequencers
• No PROTOs, No Scripts
Sample Presentation
• Simple X3D Interactive demos • Data from http://pubimage.hcuge.ch/• Volume to Surfaces by OSIRIX
6-DOF Camera
• Animation includes coordination of transformation graph and behavior graph
• Endoscopy path example:– Timesensor– PositionInterpolator, OrientationInterpolator– Routes [event model]
Specification Plan:Med X3D Profile
1. Specify additional nodes and components for X3D abstract spec
2. Create MedX3D Profile for X3D abstract spec3. Encoding» Create DICOM encoding of MedX3D Profile » Create Binary encoding of MedX3D Profile
X3D Encodings and Bindings
http://www.x3dgraphics.com
Equivalent scenegraphs
can be: encoded inand manipulated by multiple languages
X3D Components and Profiles
• Components– Families of related nodes
• Levels– Of support (fields and
functionality)
• Profiles– Well-defined nodesets
for vertical application functionality
See: http://www.web3d.org/x3d/specifications/ISO-IEC-19775-1.2-X3D-AbstractSpecification/index.html
X3D Volume Rendering Component
Specified and developed w/ US Army TATRC; two implementations (one open-source)
• General transfer function• Shading styles
– Blinn-Phong(including advanced features such as shadows and light scattering)– Cartoon shading– Gooch shading(tone mapped)
• Enhancement styles (enables enhancement of important features in an easy way)– Boundary enhancement– Edge enhancement– Silhouette enhancement
• Maximum , Minimum and Average intensity projection• Iso-surface rendering• Segmented data rendering (render different parts of volume data using different
styles)
WG 11 Direction for n-D PS
• Functional Profile (nodeset): Med X3D– X3D Interactive+ – Volumes & Rendering styles– Cutting/ clipping (i.e. for
Multiplanar Reconstruction (MPR))
• Scenegraph could be encoded in X3D Binary Encoding scenegraph profile in DICOM
Specification: Include Base X3D Concepts
• Include some set of components for meeting requirements:– Grouping: Transformation, Switch, LOD– Viewpoints (Perspective and Ortho)– Lights (Directional, Point, Spot)– 3D, 2D surfaces, lines and point geometries
• including Shape Appearance and Materials• Per-vertex data such as RGBA and ImageTexture maps
– Annotation (Text, Layers)– Interactivity
• Sensors• Event model, animation• Scripting?
Specification Progress: Volumetric Presentation
• Volume Rendering– MedX3D styles have necessary and sufficient
coverage• Masks, subvolumes & segments– MedX3D voxel data structure and access per WG2
and WG12 requirements• Addressing voxels within volume• Volume time series
• UI (Toshiba input) – e.g. Clipping & MPR
• Lines generate plane• Planes• 3D volume
DICOM WG 11 + Web3D Medical WGSpecification Plan:
1. Med X3D Functional Specification (abstract)» Specify additional nodes and components for X3D abstract
spec» Create MedX3D Profile for X3D abstract spec
1. X3D (XML, Binary) encodings exist; DICOM encoding of MedX3D Profile?
Going Forward
• ISO scenegraph technology yields a quantum leap in health care interoperability
• Immediate value in cross-platform transmission and rendering of medical image presentations
• Continue development of DICOM+MedX3D Profile: specification (X3D 3.3) including annotations, implementations, conformance examples
Ways to get Engaged
• WG 11• IEEE VR 2010 in Waltham, MA this month• Web3D Symposium in Los Angeles in July 2010• SIGGRAPH in Los Angeles in July 2010