Upload
truongmien
View
246
Download
0
Embed Size (px)
Citation preview
Automated Kidney Segmentation In 3D Ultrasound Imagery,
and its Application in Computer-assisted Trauma Diagnosis
Final PhD Oral Examination - Presentation SlidesSeptember 1st , 2016
Mahdi Marsousi
Supervisor: Prof. Konstantinos N. Plataniotis
Introduction
Abdominal trauma: injuries to abdomen, either blunt or penetrating, resulting
in severe blood loss.
Importance: massive internal bleeding quickly threatens patient’s life.
Cure: rapid diagnosis and surgery
[1]
[1] http://www.radiologyassistant.nl/en/p466181ff61073/acute-abdomen-role-of-ct-in-trauma.html
[2] http://www.wikidoc.org/index.php/Abdominal_trauma
[2]
Slide #1
Diagnosis by 3D ultrasound (3DUS) For hemodynamically unstable patients, ultrasound imagery is the preferred
imaging modality, because it is portable.
This is called: Focused Assessment with Sonography for Trauma “FAST”.
2D ultrasound is the popular diagnostic tool.
3DUS imagery is key to design computer-assisted diagnosis, because:
a) detecting and localizing internal organs in 3D space is possible by 3DUS;
b) 3DUS facilitates measuring the volume of internal bleeding.
Slide #2
:Focus of this presentation
FAST exam Six major abdominal views associated with FAST exam.
For computer-assisted trauma diagnosis, we focus on the
Morison’s pouch view (2),
• Because it is the most sensitive view to abdominal bleeding.
• An abdominal bleeding usually places between the right kidney and liver [1].
• Morison’s pouch view shows the right kidney and a portion of liver [1].
• The kidney has a unique shape among internal organs.
• By segmenting the kidney shape, we only look for an abdominal bleeding around
the top surface of the kidney shape.
[1] Heller et al. Ultrasound use in trauma: the fast exam. Academic Emergency Medicine, 14(6):525–525, 2007.
[2] Ingeman et al. Emergency physician use of ultrasonography in blunt abdominal trauma. Academic Emergency
Medicine, 3(10):931–937, 1996.
Slide #3
Motivation
Problem definition:
• Because of limited surgical resources in massive causalities, only true trauma patients should be identified and transported for rapid surgery.
• This importance necessitates the use of paramedics to conduct FAST exam.
• However, paramedics are not capable to do FAST exam with supervision.
Solution: a computer-assisted tool can be used to help paramedics to conduct
trauma diagnosis.
Slide #4
Contributions
Slide #5
3DUS dataset[1]
Slide #6
Training set: 6 with-kidney and 6 without-kidney images.
Evaluation set: 15 with-kidney and 15 without-kidney from healthy volunteers,
and 8 with-kidney images from abnormal patients.
Images of abnormal patients represent morphological changes to the kidney shape.
With-kidney image:
taken from Morison’s pouch view
Without-kidney image:
taken from other views.
[1] G. Sakas, Phase I: Ultrasound Imaging Data Collection and Applications," MedCom GmbH,
Tech. Rep., September 2012.
Contribution 1:Kidney detection
Overview on kidney detection Kidney detection: searching within 3D image domain to answer:
• whether kidney shape exists in a 3DUS image?
• if yes, what is its alignment?
State-of-the-art:
Proposed solutions:
Name Methodology Advantage Disadvantage
Noll et al. [1]
- applies volume enhancement - uses redial rays to find kidney shape mass-center
robust against speckle & low-contrast intensity
may wrongly detect other structures instead of kidney
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc. Translational Research in Medical Imag. Springer, 2014, pp. 83–90.
Slide #7
Definition: mathematical representation of kidney shape to add shape prior in kidney detection
Proposed shape modeling: complex-valued implicit shape model (CVISM)
Ψ Ԧ𝑋 = 𝜓𝑃𝑆Ԧ𝑋 + 𝜓𝐾𝐶
Ԧ𝑋 − 𝑖 ⋅ 𝜓𝑅𝑀Ԧ𝑋 ,
𝜓𝑝𝑠, 𝜓𝐾𝐶, and 𝜓𝑅𝑀 are real-positive functions, defining voxel’s membership to
Pyelocalyceal System, Kidney Capsule, and Renal Medulla, respectively.
Generating regions: using ground truth data of training set of with-kidney images.
Kidney shape model
Slide #8
Kidney
Capsule
Renal
Medulla
Pyelocalyceal
System
Real values are used to
model bright regions:
𝐾𝐶: Kidney Capsule
𝑃𝑆: Pyelocalyceal System
Imaginary values are used
to model dark region:
𝑅𝑀: Renal Medulla
Objective: registering kidney CVISM on 3DUS images, to detect kidney shape and estimate its alignment.
Challenges:
• Partial occlusion of kidney shape
• Non-kidney structures with similar appearance to the kidney shape
Components:
• Global deformation,
• Similarity metric,
• optimization algorithm.
Global deformation: similarity transformation (shape-preserving)
Ԧ𝑌 = 𝑆𝑇 Ԧ𝑝𝑠𝑡,1Ԧ𝑋 + Ԧ𝑝𝑠𝑡,2
Shape-to-volume registration
Slide #9
Orientations & scaling
Ԧ𝑝𝑠𝑡,1 = 𝜃𝑥,𝜃𝑦,𝜃𝑧,𝑠𝑇
Translations
Ԧ𝑝𝑠𝑡,2 = 𝑡𝑥,𝑡𝑦, 𝑡𝑧𝑇
CVISM domain Image domain
Proposed similarity metric: Regularized complex normalized cross-correlation (RCNCC)
Formulation:
Γ Ԧ𝑝𝑠𝑡,1, Ԧ𝑝𝑠𝑡,2∗ = max
Ԧ𝑝𝑠𝑡,2∗ ∈ΩV
𝑠𝑢𝑏
1
Λ Ԧ𝑝𝑠𝑡,1, Ԧ𝑝𝑠𝑡,2⋅max 0,ℜ Σ𝐼 Ԧ𝑝𝑠𝑡,1, Ԧ𝑝𝑠𝑡,2
Σ𝐼𝐼ℜ Ԧ𝑝𝑠𝑡,1
⋅max 0,ℑ Σ𝐼 Ԧ𝑝, Ԧ𝑋,𝑉
Σ𝐼𝐼ℑ Ԧ𝑝𝑠𝑡,1
Λ Ԧ𝑝𝑠𝑡,1, Ԧ𝑝𝑠𝑡,2−1
increases as kidney CVISM aligns out of the ultrasound field of view.
RCNCC similarity metric
Regularization Term
Checking similarity of image data with Kidney Capsule and
Pyelocalyceal System
Checking similarity of image data with Renal Medulla
Robustness againstPartial kidney shape occlusion
Considering multi-regional structure of the kidney shape,
max 𝑎, 𝑏 : selects maximum of 𝑎 and 𝑏. “max”s are used to avoid multiplication of two negative values in RCNCC. ℜ{𝑎} and ℑ{𝑏} extract real part of 𝑎 and imaginary part of 𝑏.
Slide #10
Registration problem Objective: Finding Ԧ𝑝𝑠𝑡,1
∗ and Ԧ𝑝𝑠𝑡,2∗ which provides a maximum RCNCC for input image.
Optimization problem:
Γ∗, Ԧ𝑝𝑠𝑡,1∗ , Ԧ𝑝𝑠𝑡,2
∗ = maxԦ𝑝𝑠𝑡,1
Γ Ԧ𝑝𝑠𝑡,1, Ԧ𝑝𝑠𝑡,2∗
Optimization algorithm:
• initialization: searching for best seed point from a
set of seed points to initialize registration.
• iterative improvement: iteratively updating
registration parameters using Gradient Descent.
Figure shows RCNCC metric versus 𝜃𝑥 and 𝜃𝑧.Yellow regions correspond to desirable registration solutions. The gray stars are seed point.Red star is selected seed point.Black arrows show iterations of Gradient descent.
Slide #11
Processing pipeline of kidney detection
• Γ𝑡ℎ is threshold on similarity metric to decide whether the kidney shape exists or not.
• It is obtained using training set, as Γ𝑡ℎ = 3.5.
Slide #12
Experimentation Objective: to evaluate shape-based and atlas-based kidney detection, compared to Noll et al. [1].
Dataset:
• 15 with-kidney and 15 without-kidney images of healthy volunteers
• 8 with-kidney images abnormal patients
Comparison metrics:
• accuracy: 𝐴𝑐𝑐𝐾𝐷 = 100% ⋅𝑁𝑇𝑃+𝑁𝑇𝑁
𝑁𝑇𝑃+𝑁𝐹𝑁+𝑁𝑇𝑁+𝑁𝐹𝑃,
• sensitivity: 𝑆𝑒𝑛𝑠𝐾𝐷 =NTP
𝑁𝑇𝑃+𝑁𝐹𝑁,
• specificity: 𝑆𝑝𝑒𝑐𝐾𝐷 =𝑁𝑇𝑁
𝑁𝑇𝑁+𝑁𝐹𝑃.
𝑁𝑇𝑃, 𝑁𝑇𝑁, 𝑁𝐹𝑃, and 𝑁𝐹𝑁 are numbers of true positive, true negative, false positive, false negative detections.
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based
Proc. Translational Research in Medical Imag. Springer, 2014, pp. 83–90.
Slide #13
Kidney detection accuracy Using with-kidney and without-kidney images of healthy volunteers:
• Shape-based and atlas-based provide higher accuracy, compared to Noll et al., because:
1) higher specificity: involving multi-structural regions of kidney shape,
2) higher sensitivity: regularization factor provides robustness against kidney shape’s occlusion.
• Atlas-based shows higher accuracy than shape-based because of using texture information.
Using actual ultrasound volumes of abnormal patients:
Accuracy is reduced, because:
different device settings from healthy volunteers,
Morphological changes of RUQ view in abnormal patients.
Method 𝑵𝑻𝑷 𝑵𝑻𝑵 𝑵𝑭𝑷 𝑵𝑭𝑵 Accuracy (%) Sensitivity Specificity
Noll et al. 9 11 4 6 66.67 0.60 0.73
Shape-based kidney detection 14 12 3 1 86.67 0.93 0.80
Atlas-based kidney detection 15 13 2 0 93.33 1 0.87
Slide #14
Method 𝑵𝑻𝑷 𝑵𝑭𝑵 Accuracy (%)
Noll et al. [1] 5 3 62.5
Shape-based 5 3 62.5
Atlas-based 6 2 75.0
Examples of kidney detection
Blue region: Renal Medulla
Red region: Kidney Capsule/Pyelocalyceal System
Slide #15
Contribution 2:Automated kidney segmentation
Overview on automated kidney segmentation
Problem definition: automatically segmenting the kidney shape in 3DUS images.
State-of-the-art:
Proposed method: complex-valued regional level-set with shape prior (CVRLS-SP)
• Complex-valued structure is used to add multi-regional structure into segmentation.
• Adding multi-regional structure is important because it improves segmentation specificity.
Name Methodology Advantage Disadvantage
Noll et al. [1]
Radial ray trace + fast marching + edge-based level-set
- Robust against speckle and low-contrastintensity profile
- Automated initialization
- Not robust against kidney deformation- High-computational cost
MRF-AC[2]
2D active contour + Markov random field (MRF) + 3D reconstruction
- Low computational cost - Manual initialization- Discontinuity along z-axis
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based Proc.
Translational Research in Medical Imag. Springer, 2014, pp. 83–90.
[2] Martn-Fernndez et al., An approach for contour detection of human kidneys from ultrasound images using markov random fields and active contours, Med. Image Anal., 1–23, 2005.
Slide #16
CVRLS-SP representation
Mathematical representation of regions in CVRLS-SP:
𝑏𝑟 : 𝑋 ∈ 𝑝𝑦𝑒𝑙𝑜𝑐𝑎𝑙𝑦𝑐𝑒𝑎𝑙 𝑠𝑦𝑠𝑡𝑒𝑚 𝑜𝑟 𝑘𝑖𝑑𝑛𝑒𝑦 𝑐𝑎𝑝𝑠𝑢𝑙𝑒, 𝑖𝑓 ℜ 𝜙 𝑋 > 0,
𝑑𝑟 : 𝑋 ∈ 𝑟𝑒𝑛𝑎𝑙 𝑚𝑒𝑑𝑢𝑙𝑙𝑎, 𝑖𝑓 ℑ 𝜙 𝑋 > 0,
𝑏𝑔 : 𝑋 ∈ 𝑏𝑎𝑐𝑘𝑔𝑟𝑜𝑢𝑛𝑑, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
where 𝜙 is level-set function.
Prior shape: (𝜙𝑠) is generated by aligning CVISM on detected kidney shape:
𝜙𝑠 𝑋 = 2 Ψ 𝑆𝑇 Ԧ𝑝𝑠𝑡,1∗ 𝑋 + Ԧ𝑝𝑠𝑡,2
∗ > 0 − 1 + 𝑖 , ∀𝑋 ∈ Ω𝑉
where 𝑆𝑇 Ԧ𝑝𝑠𝑡,1∗ 𝑋 + Ԧ𝑝𝑠𝑡,2
∗ places kidney CVISM, Ψ, on detected kidney shape.
level-set function, 𝜙, is initialized by prior shape: 𝜙 𝑋; 𝑡 = 0 = 𝜙𝑠 𝑋 , 𝑡 is iteration.
Slide #17
CVRLS-SP evolution
[1] Chan and Zhu. Level set based shape prior segmentation. IEEE CVPR, vol 2, pp 1164–1170, 2005..
Slide #18
Conventional regional level-set with shape prior defines energy function as [1],
𝐹 𝑐, 𝜙, 𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓, 𝑉 = 𝜆 ⋅ 𝐹𝑆𝑃 𝜙,𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓
+ 𝜇 ⋅ 𝐹𝑖𝑛𝑡 𝜙 + 𝛾 ⋅ 𝐹𝑒𝑥𝑡(𝑐, 𝜙, 𝑉)
𝐹𝑆𝑃 𝜙,𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓: shape prior energy,
𝐹𝑖𝑛𝑡 𝜙 : internal energy, controlling smoothness.
𝐹𝑒𝑥𝑡(𝑐, 𝜙, 𝑉): external energy, pushing level-set toward region of interest.
𝑐: average intensity level of region of interest, 𝐴𝑇 Ԧ𝑝𝑎𝑓: affine transformation.
𝜆, 𝜇, 𝛾: Lagrange multipliers
Contribution: CVRLS-SP’s energy functional is defined as follows,
𝐹 𝑐𝑏𝑟 , 𝑐𝑑𝑟 , 𝜙, 𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓, 𝑉 = 𝜆 ⋅ 𝐹𝑆𝑃
𝑏𝑟 ℜ{𝜙},ℜ{𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓} + 𝜆 ⋅ 𝐹𝑆𝑃
𝑑𝑟 ℑ{𝜙}, ℑ{𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓}
+𝜇 ⋅ 𝐹𝑖𝑛𝑡 𝜙 + 𝛾 ⋅ 𝐹𝑒𝑥𝑡𝑏𝑟 (𝑐𝑏𝑟 , ℜ{𝜙}, 𝑉) + 𝛾 ⋅ 𝐹𝑒𝑥𝑡
𝑑𝑟 (𝑐𝑑𝑟 , ℑ{𝜙}, 𝑉)
• Multi-regional segmentation is added By dividing both FSP and Fext into br and dr terms.
CVRLS-SP segmentation procedure
Initialization: Creating prior shape and level-set initialization
Iterative evolution:
𝜙 is updated by reducing 𝐹 𝑐𝑏𝑟 , 𝑐𝑑𝑟 , 𝜙, 𝜙𝑠 𝐴𝑇 Ԧ𝑝𝑎𝑓, 𝑉 , using Euler-Lagrange equation.
convergence criteria: ΣΔ𝜙 𝑡 = ∭ 𝜙 𝑋, 𝑡 − 𝜙 𝑋, 𝑡 − 12𝑑𝑥𝑑𝑦𝑑𝑧
Slide #19
Experimentation Objective: to evaluate proposed kidney segmentation compared to Noll et al. [1] and MRF-AC. [2]
Dataset: evaluation set of actual ultrasound volumes
Comparison metrics:
• Dice’s similarity coefficient (DSC)=2𝑇𝑃
2𝑇𝑃+𝐹𝑁+𝐹𝑃,
• Accuracy metric (ACC)=𝑇𝑃+𝑇𝑁
𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁× 100% ,
• Sensitivity measure (SENS)=𝑇𝑃
𝑇𝑃+𝐹𝑁,
• Specificity measure (SPEC)=𝑇𝑁
𝑇𝑁+𝐹𝑃.
Parameter setting:
[1] Noll et al. Automated kidney detection and segmentation in 3d ultrasound, in Clinical Image-Based
Proc. Translational Research in Medical Imaging, Springer, 2014, pp. 83–90.
[2] Martin-Fernandez et al., An approach for contour detection of human kidneys from ultrasound images
using Markov random fields and active contours, Medical Image Analysis, 1–23, 2005.
Slide #20
Name 𝜆 𝜇 𝛾 𝑁𝑖𝑡𝑟 𝑡𝑚𝑎𝑥 𝜖
Value 1 0.1 0.05 5 50 10
Specificity of CVRLS-SP is higher because:
• capability of segmenting multi-regional structure,
• using shape prior.
Box-plot: shows statistical variation of DSC of the methods:
Kidney segmentation accuracy
Method𝑫𝑺𝑪 𝑨𝑪𝑪 𝑺𝑬𝑵𝑺 𝑺𝑷𝑬𝑪
𝜇 𝜎 𝜇 𝜎 𝜇 𝜎 𝜇 𝜎
CVRLS-SP 0.8143 0.0408 97.48 0.72 0.7863 0.0814 0.9890 0.0060
Noll et al. 0.4207 0.0795 85.48 2.77 0.7024 0.0912 0.8678 0.0326
MRF-AC 0.5921 0.1457 93.10 2.74 0.6947 0.2132 0.9490 0.0246
𝜇: mean, 𝜎:standard deviation
Slide #21
CVRLS-SP MRFAC Noll et al.
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Dic
e's
Coeffic
ient(D
SC
)
Examples of kidney segmentation
Fourth column:Yellow, red, and green colors are TP, FP, and FN regions. Slide #22
Original image Detected kidney Segmented
kidney regions
Comparing
automated segmentation
vs. ground-truth
3D view
Second and third columns:
Blue and red regions indicate Renal Medulla and
Pyelocalyceal System/Kidney Capsule, respectively.
Conclusions & Future Work
Conclusions:
• We discussed the importance of automated kidney segmentation in designing a computer-
assisted trauma diagnosis.
• Shape-based and atlas-based kidney detection methods were introduced.
• Automated kidney segmentation was introduced
• The accuracy of kidney detection and segmentation methods were evaluated and compared to
state-of-the-art
• Experimental results validate the superiority of the proposed methods.
Future work:
• Investigating the effect of motion artifact and interference using a bigger dataset
• Using the automated kidney segmentation to automatically detect free fluids.
Slide #23
Publications Journal Papers:
[1] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “An Automated Approach for Kidney Segmentation in Three-Dimensional Ultrasound Images”, IEEE Journal of Biomedical and Health Informatics, DOI: 10.1109/JBHI.2016.2580040, (Date of publication: June 13th 2016)
[2] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “Computer-Assisted 3D Ultrasound Probe Placement for Emergency Healthcare Applications”, IEEE Transactions on Industrial Informatics, DOI: 10.1109/TII.2016.2569522, (Date of publication: May 18th 2016)
[3] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “Kidney Detection in 3D Ultrasound Imagery Based on Shape and Texture Priors”, to be submitted at IEEE Transaction on Biomedical Engineering (T-BME) on September 2016.
Conference Papers:
[4] M. Marsousi, X. Lee, and K. Plataniotis, “Shape-included Label-Consistent Discriminative Dictionary Learning: An Approach to Detect and Segment Multi-Class Objects in Image”, IEEE international conference on Image Processing (ICIP), 2016, accepted.
[5] M. Marsousi, K. Plataniotis, and S. Stergiopoulos. “Atlas-based segmentation of abdominal organs in 3D ultrasound, and its application in automated kidney segmentation.”, IEEE International conference in Engineering in Medicine and Biology Society (EMBC), pp. 2001-2005, 2015.
[6] M. Marsousi and K. Plataniotis, “Binomial classification based on DLENE features in sparse representation: Application in kidney detection in 3D ultrasound.” IEEE International Conference in Acoustics, Speech and Signal Processing (ICASSP), pp. 1017-1021, 2015.
[7] M. Marsousi, K. Plataniotis, and S. Stergiopoulos, “Shape-based kidney detection and segmentation in three-dimensional abdominal ultrasound images”. IEEE International conference in Engineering in Medicine and Biology Society (EMBC), pp. 2890-2894, 2014.
[8] M. Marsousi, K. Plataniotis, and S. Stergiopoulos. “A multi-steps segmentation approach for 3D ultrasound images using the combination of 3D-Snake and Level-Set.” IEEE International Conference in Digital Signal Processing (DSP), pp. 1-4, 2013.
US patent:
[9] S. Stergiopoulos, P. Shek, K. Plataniotis, and M. Marsousi. “Computer aided diagnosis for detecting abdominal bleeding with 3D ultrasound imaging.” U.S. Patent Application 14/159,744, filed January 21, 2014.