Free Free-of of-charge plugin to perform segmentation in CMR 83

Validation of performance of a free-of-charge plugin for the open-source platforms to perform cardiac segmentation in magnetic resonance imagingWojciech Szychta1,B-D, Konrad Werys2,A,E, Marzena Barczuk-Falęcka3,E, Marek Postuła4,E, Łukasz A. Małek5,A,C,E-F

*Wojciech Szychta and Konrad Werys contributed equally in the paper

A - Research concept and design, B - Collection and/or assembly of data, C - Data analysis and interpretation,D - Writing the article, E - Critical revision of the article, F - Final approval of article

1. Department of Cardiology, The Children's Memorial Health Institute, Dzieci Polskich 20, 04-730, Warsaw, Poland2. Oxford Centre for Clinical Magnetic Resonance Research, John Radcliffe Hospital, Headington, Oxford OX3 9DU, United Kingdom3. Department of Pediatric Radiology, Medical University of Warsaw, Żwirki i Wigury 63A, 02-091 Warsaw, Poland4. Department of Experimental and Clinical Pharmacology, Medical University of Warsaw, Banacha 1b, 02-097 Warsaw, Poland5. Faculty of Rehabilitation, Józef Piłsudski University of Physical Education in Warsaw, Marymoncka 34, 00-968 Warsaw, Poland

Address for correspondence:

Wojciech Szychta, Department of Cardiology, The Children's Memorial Health Institute, Dzieci Polskich 20, 04-730, Warsaw, Polandemail: wszychta@wp.pl

Konrad Werys, Oxford Centre for Clinical Magnetic Resonance Research, John Radcliffe Hospital, Headington, Oxford OX3 9DU, United Kingdomemail: konradwerys@gmail.com

Marzena Barczuk-Falęcka, Department of Pediatric Radiology, Medical University of Warsaw, Żwirki i Wigury 63A, 02-091 Warsaw, Polandemail: marz.barczuk@gmail.com

Marek Postuła, Department of Experimental and Clinical Pharmacology, Medical University of Warsaw, Banacha 1b, 02-097 Warsaw, Polandemail: mpostula@wum.edu.pl

Łukasz A. Małek, Faculty of Rehabilitation, Józef Piłsudski University of Physical Education in Warsaw, Marymoncka 34, 00-968 Warsaw, Polandtemail: lukasz.a.malek@gmail.com

Received: 2018-11-07Revised: 2018-11-10Accepted: 2018-12-12Final review: 2018-12-12DOI: 10.24255/hbj/100671

Key words:

segmentation, cardiac magnetic resonance, open-source, vendor-validated commercial software, plugin

AbstractBackground

Cardiac magnetic resonance (CMR) is the gold standard for heart size, volume and mass calculation, but a dedicated software is needed to perform the segmentation process. The aim of the study was to validate the performance of an open-source CMR segmentation tool against the latest version of vendor-validated CMR42 software.

Material and methodsWe have developed an open open-source, free-of of-charge

plugin called MRHeart for the use on open-source platforms (Osirix/Horos) which allows for detailed segmentation of the heart to obtain basic parameters. We analyzed a mixed group of 20 subjects without and with various cardiac conditions, mean age 51.5 ± 14.0 years.

ResultsSubstantial agreement (ICC between 0.95 and 0.99) was

found for all the analyzed parameters except right ventricle (the RV) stroke volume (SV) and RV ejection fraction (EF), where a lower agreement was found (ICC 0.89-–0.92). The agreement between both of the two analyzed segmentation techniques did not differ from intra- and inter-observer vari-ability of segmentation analysis with MRHeart (p>0.05). The time needed to perform segmentation by means of MRHeart was longer than with CMR42 (left ventricle [LV] without mass 4.80 ± 1.13 vs. 1.84 ± 0.77 minutes, p<0.0001; LV with mass 7.37 ± 1.82 vs. 3.43 ± 1.04 minutes, p<0.0001; RV without mass 5.57 ± 1.22 vs. 2.03 ± 0.93 minutes, p<0.0001).Conclusion

MRHeart is an open-source software which can be used for CMR segmentation analysis with a high precision, which is however slightly better for the LV than the RV.

W. Szychta et al. 84

INTRODUCTIONCardiac magnetic resonance (CMR) is the gold standard for

heart size, volume and mass calculation. However, dedicated software is needed to perform the segmentation process. This requires dedicated, expensive software or the use of tools pro-vided by scanner vendors, which are available only on their work stations [1, 2]. Management decisions increasingly rely on measurements of the size and function of both the right ventricle (RV) and the left ventricle (LV) [3, 4]. CMR is gaining popularity as a diagnostic tool in cardiology due to the techni-cal advancement of the methods and increasing availability of the scanners. To our knowledge, there are no free, open-source programs dedicated to CMR heart segmentation. An accurate and recurrent tool is crucial for reliable results. Therefore, we have developed a free-of-charge, open-source plugin called MRHeart for use on open-source platforms (Osirix/Horos). It allows for detailed segmentation of the heart to obtain basic parameters such as left and right (LV/RV) ventricular size, systolic function, and mass.

The idea was to create a plugin which can be used by read-ers who do not perform a sufficient quantity of CMR studies to justify purchase of expensive dedicated software for CMR segmentation in non-clinical settings where the regulatory certification is not required, for example in research projects or for training purposes. To this end, we decided to validate performance of open-source MRHeart against the latest version of vendor-validated and multiple market cleared CMR42 seg-mentation software (CMR42, Circle Cardiovascular Imaging Inc., Calgary, Canada).

MATERIAL AND METHODS

SubjectsWe studied 20 subjects, mean age 51.5 ± 14.0 years. The

exclusion criteria were contraindications to MRI scanning and pregnancy. Thirteen subjects of the study were randomly selected from a clinical cohort of patients with indications for CMR. Seven subjects were healthy volunteers. Details are presented in Table 1.

Parameter N=20

Age in years ± SD 51.5 ± 14.0

Male sex, (%) 15 (75)

BSA ±SD 2.1 ± 0.2

Indication for CMR- healthy volunteers (testing of new mapping sequences)- post-myocardial infarction - hypertrophic cardiomyopathy - differentiation of LV hypertrophy- valvular heart disease- ventricular arrhythmia - small benign cardiac tumor- post-myocarditis

7

5122111

Table1. Baseline characteristics of the study groupEthical approval was granted for all study procedures and

all subjects gave written informed consent.

Cardiac magnetic resonanceExaminations were performed on commercially available

1.5 T scanners (GE Healthcare; Philips Medical Systems) uti-lizing ECG-gated steady-state precession. Multiple-slice data sets, parallel to the mitral valve, covering the heart in 10–14 short-axis slices, were acquired using the following imaging parameters: repetition time, 2.2 to 3.6 ms; echo time, 1.2 ms; flip angle, 64 to 79 degrees; slice thickness, 8 mm; gap 2 mm.

Two cardiologists independently analyzed CMR studies. One cardiologist was inexperienced in CMR interpretation, while the other was a fully qualified clinician with many years of experience in this field, certified by the European Society of Cardiology for the 3rd level of competency. Prior to the trial, the inexperienced physician underwent training with one-on-one instructions by the experienced physician. The training was followed by practice on 5 sample cases. During the training, each case was discussed, and one physician was given corrections and feedback from the other. Subsequently the inexperienced cardiologist analyzed all 20 cases using pro-fessional workstation CMR42 segmentation software (CMR42, Circle Cardiovascular Imaging Inc., Calgary, Canada). Then he analyzed all cases using the MRHeart plugin in Osirix in-stalled on the same workstation. Moreover, both the physicians performed another measurement using the tested plugin on 6 randomly chosen cases, with at least a 14-day interval between repeat measurements. In each study, contours were drawn for: end-diastole – LV and RV endocardial and epicardial volumes; for end-systole: LV and RV endocardial volumes. An example of the MRHeart segmentation process with a results window is shown in Figure 1. Finally, the inexperienced physician

measured time, in all of the studies, at 4 moments during the analysis. The first measured time, called LV without mass, was from the moment when the phase of the loop determining the final diastolic phase was marked to the last contour allowing one to determine the end-diastolic volume (EDV), end-systolic volume (ESV), ejection fraction (EF) and stroke volume (SV) of the LV. The second measured time, which was called LV with mass, was from the moment when the phase of the loop determining the final diastolic phase was marked to the last contour allowing one to determine the end-diastolic mass of the LV and was a continuation of the time ‘LV without mass’. The third measured time, which was called RV without mass, was measured after resetting the clock, from the moment when the phase of the loop determining the final diastolic phase

Figure 1. EDV short-axis – sample view

Free Free-of of-charge plugin to perform segmentation in CMR 85

was marked to the last contour allowing one to determine the EDV, ESV, EF and SV of the RV. The fourth measured time, called RV with mass, was measured as a continuation of the time ‘RV without mass’ to the last contour allowing one to determine the end-diastolic mass of the RV. Plugin development

The MRHeart tool was developed in Objective-C as a pl-ugin to the Osirix/Horos image viewer. In the plugin built-in Osirix/Horos region of interest (ROI) tools are used to draw the myocardial LV/RV contours. Then volumes and masses can be calculated using Simpson’s rule, as described in [5]. Statistical methods

Nominal variables were presented as absolute numbers and percentages, continuous variables in the form of means and standard deviations (SD, in the case of variables with normal distribution) or medians and interquartile range (IQR, in the case of variables with non-normal distribution). Normality of distribution was assessed with the Kolmogorov-Smirnov test. In order to assess differences between means and medians, Student's t-test for independent variables or the Mann-Whitney test for independent variables was used. Agreement between both the analyzed segmentation methods was assessed in all patients using the Bland–Altman repeatability analysis method and interclass correlation coefficient (ICC) [6]. Inter-observer and intra-observer variability analyses were performed using the same methods but on a group of 6 randomly selected patients. The critical difference was expressed as the average ± 1.96 SD. All tests were bilateral and p<0.05 was considered as statistically significant. MedCalc software version 10.0.2.0 (MedCalc, Mariakerke, Belgium) was used for statistical anal-yses.

RESULTSThe CMR studies of 20 patients were analyzed. Table 1 sum-

marizes patients’ age, gender, BSA, and indications for CMR. Substantial agreement (interclass coefficient (ICC) between

0.95 and 0.99) was found for all the analyzed parameters except RV SV and RV EF, where a lower agreement was found (ICC 0.89–0.94). The RV mass was not compared as CMR42 does not include analysis of this parameter. Mean and critical differ-ences for each of the analyzed parameters between MRHeart plugin and CMR42 software are shown in Table 2. The average

Figure 1. EDV short-axis

Figure 1. ESV short-axis

Figure 1. Final results window

W. Szychta et al. 86

Table 2. Mean and critical differences for each of the analyzed parameters between MRHeart plugin and CMR42 software.

ICC – interclass coefficient, intra – intra-observer variability, inter – inter-observer variability, LVEDVI – left ventricular end-diastolic volume index, LVESVI – left ventricular end-systolic volume index, LVSVI – left ventricular stroke volume index, LVEF – left ventricular ejection fraction, LVMI – left ventricular mass index, RVEDVI – right ventricular end-diastolic volume index, RVESVI – right ventricular end-systolic volume index, RVSVI – right ventricular stroke volume index, RVEF – right ventricular ejection fraction.

Parameter (BSA corrected)

Mean value and SD according to the reference method (CMR42 software)

Meandifference

Criticaldifference

ICC P vs. intra P vs. inter

LVEDVI (ml/m2) 66.34 ± 15.63 1.3 4.3 0.986 0.96 0.66

LVESVI (ml/m2) 26.23 ± 9.10 0.9 3.9 0.971 0.76 0.90

LVSVI (ml/m2) 40.11 ± 9.25 0.5 3.3 0.981 0.48 0.38

LVEF (%) 61.11 ± 8.13 -0.5 4.7 0.954 0.96 0.82

LVMI (g/m2) 69.79 ± 16.51 -2.7 6.2 0.967 0.34 0.85

RVEDVI (ml/m2) 74.64 ± 18.42 -0.4 6.1 0.984 0.86 0.63

RVESVI (ml/m2) 34.46 ± 11.82 0.3 5.2 0.976 0.97 0.71

RVSVI (ml/m2) 40.18 ± 9.67 -0.2 6.0 0.942 0.90 0.72

RVEF (%) 54.60 ± 8.20 0.3 7.6 0.896 0.81 0.98

difference and the critical difference for each of the analyzed parameters between MRHeart plugin and CMR42 software, i.e. +/- 1.96 SD, are also shown in Figure 2.

Agreement between both the analyzed segmentation tech-niques did not differ significantly from intra-and inter-observer variability of segmentation analysis with MRHeart (Table 2). The results of intra- and inter-observer analysis are presented in Table 3. The lowest agreement was found for RV EF and RV

Figure 2.

Free Free-of of-charge plugin to perform segmentation in CMR 87

myocardial mass (respectively: for inter-observer variability (0.856–0.893) and intra-observer variability (0.914–0.921)). Moderate strength was noted for inter-observer variability for LV SV and RV SV (0.91–0.944). Substantial agreement was found for all the remaining analyzed parameters.

The time it took the physicians to draw out specific contours was significantly shorter when vendor-validated CMR42 seg-mentation software was used for all the measured parameters (LV without mass 4.80 ± 1.13 vs. 1.84 ± 0.77 minutes (p<0.0001); LV with mass 7.37 ± 1.82 vs. 3.43 ± 1.04 minutes (p<0.0001); RV without mass 5.57 ± 1.22 vs. 2.03 ± 0.93 minutes (p<0.0001)), which is presented in Figure 3.

DISCUSSIONTo our knowledge, this is the first validation of performance

of a free-of-charge tool for open-source platforms to analyze CMR against professional, vendor-validated commercial software. The presented data suggest that results calculated with the plugin MRHeart are of the same quality as those achieved from commercial workstations and are clinically and scientifically useful. Our study demonstrates that analysis of CMR by the plugin MRHeart shows substantial agreement for evaluation of LV with vendor-validated commercial software, though measurements for RV can be less reproducible [4].

However, vendor-validated software does not give better results. Analysis of inter-observer variability of RV volume, myocardial mass, and function between measurements in axial and short axis orientation showed poor strength of agreement according to McBride (p<0.9) in two independent trials [4, 6, 7]. The same was noted for intra-observer variability (p<0.9) for

Table 3. Inter- and intra-observer variability in assessment of the analyzed parameters assessed using MRHeart

ICC – interclass coefficient, intra – intraobserver variability, inter – interobserver variability, LVEDVI – left ventricular end-diastolic volume index, LVESVI – left ventricular end-systolic volume index, LVSVI – left ventricular stroke volume index, LVEF – left ventricular ejection fraction, LVMI – left ventricular mass index, RVEDVI – right ventricular end-diastolic volume index, RVESVI – right ventricular end-systolic volume index, RVSVI – right ventricular stroke volume index, RVEF – right ventricular ejection fraction

Parameter (BSA corrected)

Inter-observer variability Intra-observer variability

Meandifference

Criticaldifference

ICC Meandifference

Criticaldifference

ICC P, intra vs. inter

LVEDVI (ml/m2) 0.6 7.2 0.976 -0.5 5.9 0.985 0.77

LVESVI (ml/m2) -1.9 3.1 0.966 0.6 3.5 0.980 0.74

LVSVI (ml/m2) 2.6 5.1 0.944 1.1 4.4 0.955 0.89

LVEF (%) 2.7 2.1 0.939 1.2 3.0 0.951 0.89

LVMI (g/m2) 0.3 4.8 0.974 -0.1 4.0 0.990 0.55

RVEDVI (ml/m2) -1.0 4.2 0.971 1.1 3.2 0.980 0.82

RVESVI (ml/m2) -2.1 3.3 0.962 0.1 4.2 0.977 0.75

RVSVI (ml/m2) 2.2 5.4 0.910 1.0 3.9 0.950 0.71

RVEF (%) 2.4 4.1 0.893 2.2 4.0 0.921 0.85

RVMI (%) -2.6 1.9 0.856 -2.0 1.5 0.914 0.74

Figure 3.

W. Szychta et al. 88

RV SV and RV EF [4, 6, 7]. The results of our study may also be influenced by the limited experience of the observer. Proper training and experience can influence time and quality of the final results [4]. Factors such as experience, occurrence of right bundle branch block (RBBB), mass and size of trabec-ulae and papillary muscles can influence the final results of RV analysis [4, 8, 9]. Underscoring of the RV performance can have serious consequences for patient management. It is well known that the timing of surgical interventions or feasibility of pregnancy can be crucial and is based on the results of RV evaluation [7, 10]. However, we have chosen an inexperienced reader to validate the software in order to show that heart segmentation can be done under well-structured guidance not only by readers experienced in CMR studies by also by radiologists or cardiologists who do not perform those studies on a daily basis. Easily accessible, open-source software for CMR segmentation will give more training opportunities for the inexperienced group of readers.

Right ventricular mass was not compared as CMR42 does not include analysis of this parameter. This limits clinical possi-bilities of this particular vendor-validated commercial software. RV myocardial mass, although not commonly analyzed in clinical settings, should be kept in mind by clinicians as it is one of the symptoms of RV remodeling (increased mass and volume). Such a correlation with impaired systolic and diastolic function has been well documented in overweight and obese children and adults [11, 12]. Moreover, when reduction in RV free wall longitudinal strain in obese/overweight children can be found despite preserved RV ejection fraction in children with LV concentric hypertrophy and reduced LV, this particular group of patients is of a high-risk phenotype with increased risk of cardiovascular disease and premature death [11, 12]. In patients with congenitally corrected transposition of the great arteries and transposition of the great arteries increased RV mass index is one of the predictors of major cardiac events [13].

Agreement between both the analyzed segmentation tech-niques did not differ from intra- and inter-observer variability of segmentation analysis with MRHeart. Greater intra- and inter-observer variability was observed in two cases. In the first one, inter-observer variability was noted as moderate for LV EF and poor for RV EF, which can be explained by the mathematical method of evaluation of EF. The other, and the most important, is the difference in RV myocardial mass. Such an observation is most likely due to RV trabeculations, thin wall and complex shape, that is, other than conical (typical for LV), regional abnormalities in the RV free wall, tethering and tricuspid regurgitation [4, 8, 14]. From the practical point of view, the difference did not influence the final results of the study as the mean difference for RV EF was 2.4%, while the critical difference was 4.1%. Moreover, analysis of LV EF shows the mean difference of 2.7% and the critical difference of 2.1%. The same was noted by Mooij et al. [4].

The time needed to perform segmentation by means of MRHeart was longer than with CMR42. Such a result was predictable and results from the differences between the applications. Analysis performed in MRHeart requires the user to undertake more actions as, besides contouring, selec-

tion of which contour is drawn (the LV or RV, endocardium or epicardium) is also needed. CMR42 allows selection of the contoured part of the heart just once. Vendor-validated commercial software has semiautomatic tools, which allow one to perform fast contouring. Such tools are very useful, especially when analyzing the RV. In the process of contour-ing, we included trabeculations and papillary muscles in the volume of the analyzed chamber. However, results may differ among physicians if different analysis protocols were applied in different medical centers. CMR42 application al-lows one to include trabeculations and papillary muscles in the automatic analysis as well as exclude them, which should not affect the time needed for the study analysis. It is worth emphasizing that the time measurement was taken after the selection of end-systole and end-diastole moments. In our study, time was measured by an inexperienced physician who most likely needed more time during the analysis than the more experienced physician. Therefore, we can state that such results represent the worst possible scenario for study duration. The learning curve of CMR shows that by the time the 120th study is processed, the time required is reduced by even as much as 45% for RV analysis and after 2 months of intense training analysis of all the parameters is improved, except for LV mass [4, 15].

For use in an everyday clinical setting, software should be cleared according to specific regulations for different markets, for example section 510(k) of the Food, Drug and Cosmetic Act (FDA). In the current form MRHeart has not been put under a regulatory process, and thus is feasible for settings that do not require medical certification, for example research studies. A research study often involves obtaining results from a small group of participants. In such a setting the purchase of expensive dedicated software for CMR segmentation may not be justified.

Our study has some limitations. The data were collected from only two observers, and therefore further research is needed to support our results. One of the observers was inexperienced. However, properly trained beginners can achieve reliable results. An inexperienced observer may tend to underestimate volumes of LV EDV and LV ESV and over-estimate LV mass before teaching [15]. We did not analyze such correlations in our study as this was not the aim of the study. The population analyzed in our study was small, but the population size allowed conclusions to be drawn from the statistical analysis.CONCLUSION

MRHeart is open-source software which can be used for CMR segmentation analysis with high precision, but slightly better for the LV than the RV.

References

1. Pennell D. Cardiovascular magnetic resonance: twen-ty-first century solutions in cardiology. Clin Med 2003;3:273-8.

Free Free-of of-charge plugin to perform segmentation in CMR 89

2. Alfakih K, Plein S, Thiele H, Jones T, Ridgway JP, Siva-nanthan MU. Normal human left and right ventricular dimensions for MRI as assessed by turbo gradient echo and steady-state free precession imaging sequences. J Magn Reson Imaging 2003;17:323-9.

3. Ponikowski P, Voors AA, Anker SD, et al. 2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC). Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur Heart J 2016;37:2129-200.

4. Mooij CF, de Wit CJ, Graham DA, Powell AJ, Geva T. Reproducibility of MRI measurements of right ven-tricular size and function in patients with normal and dilated ventricles. J Magn Reson Imaging 2008;28:67-73.

5. Reiter G, Reiter U, Rienmüller R, Gagarina N, Ryabikin A. On the value of geometry-based models for left ven-tricular volumetry in magnetic resonance imaging and electron beam tomography: a Bland–Altman analysis. European Journal of Radiology 2004;52:110-118.

6. McBride G. A proposal for stenght-ofagreement criteria fir Lin.s.Concordance correlation coefficient. National Institute of Water & Atmospheric Research Ltd Ham-ilton, NIWA, New Zealand https://www.medcalc.org/download/pdf/McBride2005.pdf; 2005 [accessed 07 April 2018].

7. van der Bom T, Romeih S, Groenink M, et al. Evalu-ating the systemic right ventricle by cardiovascular magnetic resonance: short axis or axial slices? Congenit Heart Dis 2015;10:69-77.

8. Chuang ML, Gona P, Hautvast GL, Salton CJ, Blease SJ, Yeon SB, et al. Correlation of trabeculae and papillary muscles with clinical and cardiac characteristics and impact on CMR measures of LV anatomy and function. JACC Cardiovasc Imaging 2012;5:1115-23.

9. Marterer R, Hongchun Z, Tschauner S, Koestenberger M, Sorantin E. Cardiac MRI assessment of right ven-tricular function: impact of right bundle branch block on the evaluation of cardiac performance parameters. Eur Radiol 2015;25:3528-35.

10. Drenthen W, Pieper PG, Ploeg M, et al. Risk of com-plications during pregnancy after Senning or Mustard (atrial) repair of complete transposition of the great arteries. Eur Heart J 2005;26:2588-95.

11. Jing L, Pulenthiran A, Nevius CD, et al. Impaired right ventricular contractile function in childhood obesity and its association with right and left ventricular

changes: a cine DENSE cardiac magnetic resonance study. J Cardiovasc Magn Reson 2017;19:49.

12. Jing L, Binkley CM, Suever JD, et al. Cardiac remod-eling and dysfunction in childhood obesity: a cardio-vascular magnetic resonance study. J Cardiovasc Magn Reson 2016;18:28.

13. Shiina Y, Inai K, Takahashi T, et al. Inter- and intra-ven-tricular dyssynchrony in the systemic right ventricle is a surrogate marker of major cardiac events in mildly symptomatic patients. Heart Vessels 2018; in press. doi: doi: 10.1007/s00380-018-1144-2.

14. Medvedofsky D, Addetia K, Patel AR, et al. Novel ap-proach to three-dimensional echocardiographic quanti-fication of right ventricular volumes and function from focused views. J Am Soc Echocardiogr 2015;28:1222-31.

15. Hedstrom E, Ishida M, Sepulveda-Martinez A, et al. The effect of initial teaching on evaluation of left ventricular volumes by cardiovascular magnetic res-onance imaging: comparison between complete and intermediate beginners and experienced observers. BMC Med Imaging 2017;17:33.