Utility of Machine Learning Algorithms in Assessing Patients with a Systemic Right Ventricle

Gerhard-Paul Diller MD PhD MSc (1,2,3), Sonya Babu-Narayan MB BBS BSc PhD (1), Wei Li MD PhD (1), Jelena Radojevic MD (1,5), Aleksander

Kempny MD (1), Anselm Uebing MD PhD (1,3), Konstantinos Dimopoulos MD MSc PhD FESC (1), Helmut Baumgartner MD FESC FACC (2,3), Michael A Gatzoulis MD PhD FESC FACC (1), Stefan Orwat MD (2,3)

(1) Adult Congenital Heart Disease Programme, Royal Brompton Hospital, London, UK

(2) Dept. of Cardiology III – Adult Congenital and Valvular Heart Disease, University Hospital Muenster, Muenster, Germany

(3) Competence Network for Congenital Heart Defects, DZHK (German Centre for Cardiovascular Research), Berlin, Germany

(4) Division of Paediatric Cardiology, University Hospital Muenster, Germany.

(5) Cardiologie Congenitale, Strasbourg, France

Conflict of Interest: none

Correspondence to:Dr Gerhard-Paul DillerDepartment of Cardiology IIIAdult Congenital and Valvular Heart Disease, University Hospital Muenster,Albert-Schweitzer Campus 1Muenster, GermanyTel. +49 251 83 46138Fax. +49 251 83 46109E-mail: gerhard.diller@ukmuenster.de

Word count: 4294 words

Abstract

Aims: To investigate the utility of novel deep learning (DL) algorithms in recognizing transposition of the great arteries (TGA) after atrial switch procedure or congenitally corrected TGA (ccTGA) based on routine transthoracic echocardiograms. In addition, the ability of DL algorithms for delineation and segmentation of the systemic ventricle was evaluated.

Methods and Results: 132 patients (92 TGA and atrial switch and 40 with ccTGA; 60% male, age 38.3±12.1 years) and 67 normal controls (57% male, age 48.5±17.9 years) with routine transthoracic examinations were included. Convolutionalneural networks were trained to classify patients by underlying diagnosis and a U-Net design was used to automatically segment the systemic ventricle.Convolutional networks were build based on over 100,000 frames of an apical 4-chamber or parasternal short axis view to detect underlying diagnoses. The DL algorithm had an overall accuracy of 98.0 % in detecting the correct diagnosis. The U-Net architecture model correctly identified the systemic ventricle in all individuals and achieved a high performance in segmenting the systemic right or left ventricle (Dice metric between 0.79 and 0.88 depending on diagnosis) when compared to human experts.

Conclusion: Our study demonstrates the potential of machine learning algorithms, trained on routine echocardiographic datasets to detect underlying diagnosis in complex congenital heart disease. Automated delineation of the ventricular area was also feasible. These methods may in future allow for the longitudinal, objective and automated assessment of ventricular function.

Abstract length: 232 Words

Introduction

Patients with congenital heart disease represent a growing cohort with lifelong

chronic heart disease. These patients require longitudinal follow-up at specialized centres.1

Over time, especially patients with a systemic right ventricle (RV) are at risk of developing

progressive RV dysfunction2-4 and heart failure5 with associated morbidity and mortality.6, 7

As a consequence, early detection of myocardial dysfunction and objective assessment of RV

function are paramount. Regular transthoracic echocardiographic investigations remain the

main imaging modality for detecting complications and guiding therapy in this patient

population.8 In addition, undiagnosed patients with congenitally corrected transposition of the

great arteries (TGA) are occasionally identified as part of a routine echocardiographic

investigation in adulthood.9 Even experienced echocardiographers without major exposure to

adult congenital heart disease may occasionally struggle to make the correct diagnosis due

the complex anatomy involved in patients with a systemic RV. Given the dramatic progress

in the field of machine learning, especially involving deep learning/neuronal networks over

the last few years,10-13 we aimed to test the utility of this technology in complex adult patients

with a systemic right ventricle. Specifically, we hypothesized that DL networks would be

able to discriminate between patients with TGA after atrial switch operation, patients with

congenitally corrected TGA (ccTGA) and normal controls. Expanding on this, we also aimed

to develop methods for automatic recognition of the systemic ventricle and area

segmentation, which may be useful for objective longitudinal assessment of ventricular

function in the long-term follow-up of these patients.

Patients and Methods

We identified patients with a systemic RV who attended the Adult Congenital Heart

Disease Programme, Royal Brompton Hospital, London and the Adult Congenital and

Valvular Heart Disease Centre at the University of Muenster, Germany and had undergone a

transthoracic echocardiogram between 2005 and 2018. In addition, subjects with structurally

normal hearts were identified and served as normal controls. We excluded patients who

lacked adequate echocardiographic sequences. Echocardiographic recordings, performed

according to current recommendations,14, 15 of the apical 4-chamber and parasternal short axis

view stored in DICOM format were retrieved. The cine loops were split into individual

frames and converted to a bitmap format using a dedicated MatLab (Version R2018a)

program. Patient identifying information and metadata had been removed before analysis.

Retrospective analysis of the de-identified echocardiographic recordings has been approved

by the local ethics committee.

Model architecture and training for diagnostic classification

Image frames were categorized by diagnosis and the dataset was split into a

training/validation (80%) and test group (20%). Frames from test group patients were not

used for training to ensure that the external validity of the model - when exposed to new

patients - could be quantified. The image resolution was sampled down to 150x150 pixels

with 256-shaded greyscale pixel values. Image augmentation with random application of

rotations (±10°), width and height shifts (10 and 5%, respectively) as well as shears and zoom

(up to 10% and 5%, respectively) were applied to the frames at run-time. The underlying

neuronal network is illustrated in Figure 1. It is based on a series of convolution layers (3x3

size), max pooling layers (2x) and fully connected layers at the end. In addition, drop-out

layers were utilized to avoid overfitting. Rectified linear units were used as activation

functions, except for the last layer being a softmax classifier layer.

The model was trained over 40 epochs with RMSprop optimization and a batch size of 32.

Hyperparameters were adjusted to ensure maximal accuracy while avoiding overfitting based

on inspection of the convergence plots for training and validation accuracy/loss. Accuracy

was assessed by quantifying the percentage of correctly classified frames.16 To illustrate

learning representations, we extracted and displayed individual activation of intermediate

convolutional layers. Networks were implemented based on Tensorflow using the keras

package for R.17 Training and testing was performed on an Intel i7 platform with GPU

support (Nvidia GX 1070). Analyses were performed using RStudio Version 1.1.456 / R-

package version 3.5.1. Training of the network described above required approximately 30-

45 minutes.

Model architecture and training for image segmentation

Image segmentation was performed based on the U-Net architecture implemented in R/keras.

The details of the U-Net have been described previously.18 Briefly, it consists of

convolutional layers with rectified linear and max pooling units forming a contracting path

with the addition of a symmetrical, expanding path consisting of up-convolutional layers and

concatenations of features from the contracting path. The model was previously specifically

developed for biomedical image analysis. For full details see https://lmb.informatik.uni-

freiburg.de/people/ronneber/u-net/. A random subsample of approximately 1,200 ccTGA,

TGA and normal subject frames were all manually annotated (by GD and SO), providing a

mask for the systemic ventricle. The mask delineated the endocardial border during all stages

of the heart cycle in an apical 4-chamber view. The dataset was split into a training/validation

(80%) and test group (20%). Frames were down-sampled to a resolution of 128x128 pixels

before analysis. As above, we ensured that no frames from the test group were used for

training or validation of the model. Before being presented to the model, frames were

subjected to image augmentation as described above and intensity of the image frames was

normalized. It took about 180 minutes to train the network on the Nvidia GX 1070 setup.

Method evaluation was performed by assessing the Dice metric as well as relative area

difference calculated as the ratio of the area difference between predicted ventricular area and

actual ventricular area divided by the actual area (ground truth area). The Dice metric

measures the overlap between automated and manual (ground-truth) segmentation. The

metric’s value ranges between 0 and 1, with 0 denoting no overlap and 1 corresponding to

perfect agreement. A higher Dice value is indicative of better agreement.19 Relative area

difference is conceptually similar to the metrics used for assessing the agreement between

two imaging modalities. The two models (classification and segmentation) were run

independently. Combining a classification model with an appropriate segmentation model is

equally feasible; we refrained from doing so to reduce computational resource use.

Statistical analysis

Descriptive data are presented as mean±standard deviation or median and interquartile range

(IQR) for continuous variables, while categorical variables are presented as number

(percentage). Comparisons between groups were made using a 2-tailed Student t-test or

nonparametric methods depending on data distribution. Comparison between groups A two-

sided P-value of <0.05 was considered indicative of statistical significance.

Results

Overall 132 patients with a systemic RV (92 with TGA after atrial switch and 40

ccTGA) and 67 normal controls formed the basis of this study. Patients’ demographics and

clinical information are presented in Table 1. After splitting the apical 4-chamber cine loops,

73,425 TGA frames, 33,394 ccTGA frames and 24,354 frames from normal controls were

included in the analysis. In addition, 8,892 ccTGA frames, 17,053 normal frames and 37,272

d-TGA frames from a parasternal short axis view were available.

Classification into ccTGA, TGA or normal cardiac anatomy

The model for the 4-chamber apical view achieved an accuracy of 95.5% in the training set

and 94.4% in the test set (based entirely on frames from new patients, not used for model

training) on an on frame basis. The accuracy to detect the correct diagnosis was 86.2% for

individual frames of ccTGA subjects, 91.6% for frames of normal hearts and 97.5% for

frames of TGA patients. When combing all cine loop frames of one particular patient, 100%

of the TGA, 100% of the normal hearts and 93.8% of the ccTGA hearts were correctly

identified.

For the parasternal short axis 2-chamber view, the model achieved an accuracy in the training

set of 96.7% but only 76.8% in the test set. Especially, categorizing ccTGA frames correctly

proved to be challenging (accuracy 49.2% in the test set), while detecting normal hearts and

TGA subjects had better accuracy rates (85.6% and 86.8%, respectively) in the test set.

When combining apical 4-chamber and parasternal short axis views the model achieved an

accuracy of 98.0% in the test set. On an on frame basis, it correctly classified 97.0% of

ccTGA frames, 94.0% of normal frames and 98.2% of TGA frames. Requiring a class

probability of at least 90%, 75.7% of ccTGA, 83.8% of normal heart frames 94.8% of TGA

frames were correctly identified. When combining all frames from one cine loop, the model

correctly categorized diagnosis in all patients from the test set.

Visualization of the network activation

To illustrate model stimulation for different patient cohorts, we investigated the

activation of intermediate convolutional layers for ccTGA, normal and TGA

echocardiographic frames, respectively. Figure 2 illustrates the activation pattern for the last

2-D convolutional layer of the DL model, showing the dissimilar layer response depending

on underlying diagnosis. Especially, the systemic ventricle was found to display prominently

on the layer activation map.

Ventricular segmentation

Models for ventricular segmentation were trained individually for ccTGA, TGA and

normal hearts based on a training set of approximately 1000 random frames each (consisting

of a pair of original frame and manually produced endocardial mask) for the systemic

ventricle as illustrated in Figure 3. The model achieved a good recognition rate, and in fact,

the systemic ventricle was correctly identified in all frames from the test set, irrespective of

diagnosis. When assessing accuracy of area detection, the best results were obtained in

normal hearts (Dice score 0.881±0.062), followed by TGA (Dice score 0.867±0.067) and

ccTGA hearts (Dice score 0.794±0.083). The mean relative area difference between predicted

and actual mask was -2.25% (±8.4%) for normal hearts, -5.87% (±7.00%) for TGA hearts

and 1.88% (±17.20%) for ccTGA hearts, respectively (Figure 4).

Comparison to human interobserver variability

Two experienced echocardiographers (GD and SO) each performed ventricular

segmentation independently in a blinded fashion on a random subsample of 50 ccTGA, TGA

and normal hearts. When applying the same metrics as for the automatic detection algorithm

to these two datasets, the following results were obtained: Dice scores were highest for

normal hearts (0.878±0.037), followed by TGA (0.868±0.040) and ccTGA (0.817±0.049).

The mean relative area difference was -6.62% (±7.75%) for normal hearts, -11.29% (±8.55%)

for TGA hearts and -15.71%(±13.76%) for ccTGA hearts, respectively.

Discussion

The present study demonstrates that automated detection of the underlying anatomy is feasible based on 2-dimensional transthoracic echocardiographic frames acquired as part of routine care in patients with a systemic RV. The algorithm correctly identified over 95% of frames in the test cohort of patients. When including all frames acquired in a single patient no individual was misclassified by the combined apical and short axis model. The automatic segmentation model also identified the correct systemic ventricle in 100% of frames with a relative error comparable to human experts.

Progressive systemic right ventricular dysfunction is a recognized complication of TGA patients after atrial switch operation and ccTGA patients.4, 8, 20, 21 Therefore, life-long regular echocardiographic assessment is recommended and part of routine follow-up in this population. While, alternative methods such as tricuspid annular plane systolic excursion, RV strain or strain rate have been proposed,22-26 traditional RV fractional area change remains an established and widely used echocardiographic marker of systolic RV function.27 The current study illustrates how appropriate DL models can be trained to recognize the systemic ventricle even in patients with complex cardiac anatomy and delineate the endocardial border in this setting. We contend that this technology has the potential to assist non-congenital echocardiographers arriving to the correct diagnosis and may allow in the future for a semi- or fully-automated longitudinal assessment of RV size and function. Recently, Madani et al. have described a DL framework for automatic view classification of routine echocardiograms based on convolutional networks.16 This methodology could well be combined with our network to automatically split the echocardiographic recordings into individual views and subsequently use the apical 4-chamber and parasternal 2-chamber short axis view for diagnostic classification and area quantification. Recently, networks assessing the quality of echocardiographic investigations have been described and could also be incorporated in the general workflow, thus avoiding the quantification of inadequate sequences.28 As clinicians are often faced with the question of longitudinal change in systemic RV function, these tools could significantly enhance workflow and promise important efficiency gains in the follow-up of congenital heart patients. Furthermore, recent research highlighted the scientific value of longitudinal data assessment in patients with a systemic RV. This suggests that such information should be incorporated in future risk stratification models, potentially yielding superior prognostic power compared to conventional cross sectional analyses.29

The present study is – to our knowledge - the first to assess the utility of deep learning algorithms in the imaging of complex congenital heart disease. While we focused specifically on patients with systemic RV, the methods presented here could easily be extended to other

complex congenital cohorts such as Ebstein anomaly or univentricular hearts. Within a multicentre approach with appropriate funding, comprehensive DL-models for various underlying diagnoses could be established covering different imaging modalities across the entire spectrum of congenital heart disease and age groups. Applying the methods described here to cardiac magnetic resonance imaging is a logical next step and should yield superior detection rates and segmentation accuracy thanks to the better image quality of the latter. In fact, recently Bai et al. have described the results of automatic image segmentation using fully convolutional network models applied to cardiac MRI cine-images from normal controls.19 This technology is very similar to the U-Net utilized in the current study. Although, neuronal networks are commonly regarded as black boxes, our study illustrates that by interrogating the activation response of intermediate layers to various samples presented, important clues on the relevant features for recognition can be obtained. Interestingly, we found that our model activates especially on the myocardial mass of the systemic ventricle, an appropriate choice based on the underlying anatomic differences.

Patients with a systemic RV present with heterogenous anatomy and are inherently difficult to assess echocardiographically.15 Therefore, we contend that demonstrating the performance of DL algorithms in this challenging population using echocardiographic frames of varying quality highlights the potential of this technique for other groups of patients with congenital heart disease and structural cardiac defects. We can only speculate why model performance was consistently lower in ccTGA patients compared to the TGA cohort. This may be related to the fact that the anatomic variability is greater in ccTGA compared to other forms of congenital heart disease and especially the short axis in ccTGA hearts can be variable in appearance, bearing similarities to normal anatomy as illustrated in the online appendix, especially if image quality is suboptimal.

Limitations

Our study was based on retrospective imaging data from two tertiary care centres for adult congenital heart disease. Therefore, we cannot exclude the possibility that the echocardiographic image quality may be superior to that obtainable in the community. In addition, the performance of the model may be worse when applied to a different patient population. The model, therefore, requires external validation. However, the study covered a period of >10 years and image quality varied between patients and across the study period. The current study used single 2-D frames for identification of underlying diagnosis and ventricular segmentation. With increasing experience and more powerful computer technology, 3-D image analysis and networks such as 3D convolutional networks, working directly with cine loops should be addressed in future studies and may further improve

classification results. The published segmenting performance results - based on magnetic resonance imaging frames in normal subjects-19 are slightly superior to the current results. However, the echocardiographic image quality available to us was limited compared to cardiac magnetic resonance data and Dice scores as well as the area difference obtained in the current study were comparable to those obtained when assessing interobserver difference between human investigators employing our echocardiographic data. Further studies are required to assess the performance of the model when applied to different congenital heart defects not used for model training. This reinforces the need for external validation based on a large sample of congenital echocardiographic frames spanning the entire spectrum of congenital heart disease. For practical use, however, it is possible to set the model a probability threshold. This would require a certain prediction probability to be reached before making a certain diagnosis, thus, avoiding that samples with an unknown diagnosis are merely assigned to the category with the highest probability.

Conclusions

The current study demonstrates for the first time the utility of machine learning algorithms, trained on routine echocardiographic datasets, to detect underlying diagnosis in patients with complex congenital heart disease. Automatic segmentation of the ventricular area is also feasible based on appropriate DL algorithms. These methods may allow for longitudinal, objective and automated assessment of ventricular function in future and guide management.

Contributor Information

GPD and SO planned and conducted the study and are responsible for the overall content as

guarrantors. GPD and SO prepared and analysed the data using DL networks. SBN, WL, JR,

MG, AK, HB, KD and AU made substantial contributions in analysis, drafting the article and

revising it critically for important intellectual content. All authors gave final approval of the

version to be submitted and any revised version.

Funding

This study was supported by a research grant from the EMAH Stiftung Karla Voellm,

Krefeld, Germany. The Adult Congenital Heart Centre and Centre for Pulmonary

Hypertension, Royal Brompton Hospital, London, UK have received support from Actelion

UK, Pfizer UK, GSK UK, the British Heart Foundation and the NIHR Cardiovascular and

Respiratory Biomedical Research Units.

Competing interests

None declared.

Table 1. Patient demographics

Parameter Atrial switch TGA ccTGA Normals p-Value

N (% male *) 92 (55%) 40 (70%) 67 (57%) P=0.27Age 35.4 ±8.7 45.6 ± 15.9 48.5 ± 17.9 P<0.001NYHA baseline I: 75% II:23%

III:2%I: 45% II: 36% III:

18%I: 100% P<0.001

Syst.-EF (%)* 51.2±10.0% 47.5±18.0% 64.0±6.2% P<0.001Subpulm.-EF (%)* 59.5±10.8% 55.8±12.4% P=0.66Severe TR 2% 9% - P=0.08Pre-existing PM 26% 34% - P=0.56

EF=ejection fraction; PM=pacemaker; Syst.=systemic ventricle; Subpulm.=subpulmonary ventricle; TR=tricuspid regurgitation. Ejection fraction was measured on magnetic resonance imaging in patients with a systemic right ventricle; ejection fraction in the normal controls is based on echocardiographic assessment (Simpson method).

Figures:

Figure 1: Architecture of the convolutional neural network used for image classification. Frames down-sampled to a resolution of 150x150 pixels were used as input data. ccTGA = congenitally corrected transposition of the great arteries. Conv=convolutional layer. FC=fully connected layer. TGA= transposition of the great arteries.

Figure 2: Activation maps of the intermediate network layers, illustrating the predominant activation of the convolutional network based on underlying diagnosis especially in the area of the systemic ventricle. Orange, yellow and white indicate increasing local activation levels.

Figure 3: Illustration of the training process of the U-Net model based on pairs of raw image and manually provided ground-truth mask. The model is trained to provide a segmentation mask based on this information.

Figure 4: Representative examples of systemic ventricular segmentation in subjects with a structurally normal heart, transposition of the great arteries (TGA) and congenitally corrected TGA (ccTGA) in the external test cohort (upper panel). The red line represents the manually provided ground truth line, the green line is the automatically generated segmentation line by the deep learning algorithm. The lower panel illustrates the area variability between automatic segmentation and manually provided ground truth mask. The values indicate mean±standard deviation.

Figure 1:

Figure 2:

Figure 3:

Figure 4:

References

1. Warnes CA. Transposition of the great arteries. Circulation 2006;114(24):2699-709.

2. Baumgartner H, Bonhoeffer P, De Groot NM, de Haan F, Deanfield JE, Galie N et al., Task Force on the Management of Grown-up Congenital Heart Disease of the European Society of C, Association for European Paediatric C, Guidelines ESCCfP. ESC Guidelines for the management of grown-up congenital heart disease (new version 2010). Eur Heart J 2010;31(23):2915-57.

3. Buch J, Wennevold A, Jacobsen JR, Hvid-Jacobsen K, Lauridsen P. Long-term follow-up of right ventricular function after Mustard operation for transposition of the great arteries. Scand J Thorac Cardiovasc Surg 1988;22(3):197-202.

4. Roos-Hesselink JW, Meijboom FJ, Spitaels SE, van Domburg R, van Rijen EH, Utens EM et al. Decline in ventricular function and clinical condition after Mustard repair for transposition of the great arteries (a prospective study of 22-29 years). Eur Heart J 2004;25(14):1264-70.

5. Piran S, Veldtman G, Siu S, Webb GD, Liu PP. Heart failure and ventricular dysfunction in patients with single or systemic right ventricles. Circulation 2002;105(10):1189-94.

6. Diller GP, Kempny A, Alonso-Gonzalez R, Swan L, Uebing A, Li W et al. Survival Prospects and Circumstances of Death in Contemporary Adult Congenital Heart Disease Patients Under Follow-Up at a Large Tertiary Centre. Circulation 2015;132(22):2118-25.

7. Nieminen HP, Jokinen EV, Sairanen HI. Causes of late deaths after pediatric cardiac surgery: a population-based study. J Am Coll Cardiol 2007;50(13):1263-71.

8. Brida M, Diller GP, Gatzoulis MA. Systemic Right Ventricle in Adults With Congenital Heart Disease: Anatomic and Phenotypic Spectrum and Current Approach to Management. Circulation 2018;137(5):508-518.

9. Matsakas EP, Perpinia AS, Kambitsi EH, Kossyvakis HI, Hamodraka ES. Congenitally corrected transposition of the great arteries in a seventy-year-old woman. Hellenic J Cardiol 2005;46(5):370-3.

10. Al'Aref SJ, Anchouche K, Singh G, Slomka PJ, Kolli KK, Kumar A et al. Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging. Eur Heart J 2018.

11. Johnson KW, Torres Soto J, Glicksberg BS, Shameer K, Miotto R, Ali M et al. Artificial Intelligence in Cardiology. J Am Coll Cardiol 2018;71(23):2668-2679.

12. Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial Intelligence in Precision Cardiovascular Medicine. J Am Coll Cardiol 2017;69(21):2657-2664.

13. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436-44.

14. Lang RM, Bierig M, Devereux RB, Flachskampf FA, Foster E, Pellikka PA et al. Chamber Quantification Writing G, American Society of Echocardiography's G, Standards C, European Association of E. Recommendations for chamber quantification: a report from the American Society of Echocardiography's Guidelines and Standards Committee and the Chamber Quantification Writing Group, developed in conjunction with the European Association of Echocardiography, a branch of the European Society of Cardiology. J Am Soc Echocardiogr 2005;18(12):1440-63.

15. Li W, West C, McGhie J, van den Bosch AE, Babu-Narayan SV, Meijboom F, Mongeon FP et al. Consensus recommendations for echocardiography in adults with congenital heart defects from the International Society of Adult Congenital Heart Disease (ISACHD). Int J Cardiol 2018.

16. Madani A, Arnaout R, Mofrad M, Arnaout R. Fast and accurate view classification of echocardiograms using deep learning. npj Digital Medicine 2018;1(1):6.

17. Chollet F. Keras. GitHub (2015). https://github.com/keras-team/keras (last accessed 02/10/2018).

18. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. 2015, p. 234-241. Springer, Berlin.

19. Bai W, Sinclair M, Tarroni G, Oktay O, Rajchl M, Vaillant G et al. Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J Cardiovasc Magn Reson 2018;20(1):65.

20. Diller GP, Radojevic J, Kempny A, Alonso-Gonzalez R, Emmanouil L, Orwat S et al. Systemic right ventricular longitudinal strain is reduced in adults with transposition of the great arteries, relates to subpulmonary ventricular function, and predicts adverse clinical outcome. Am Heart J 2012;163(5):859-66.

21. Warnes CA. Adult congenital heart disease importance of the right ventricle. J Am Coll Cardiol 2009;54(21):1903-10.

22. Chow PC, Liang XC, Cheung EW, Lam WW, Cheung YF. New two-dimensional global longitudinal strain and strain rate imaging for assessment of systemic right ventricular function. Heart 2008;94(7):855-9.

23. Eindhoven JA, Menting ME, van den Bosch AE, McGhie JS, Witsenburg M, Cuypers JA et al. Quantitative assessment of systolic right ventricular function using myocardial deformation in patients with a systemic right ventricle. Eur Heart J Cardiovasc Imaging 2015;16(4):380-8.

24. Becker M, Humpel C, Ocklenburg C, Muehler E, Schroeder J, Eickholt C et al. The right ventricular response to high afterload: comparison between healthy persons and patients with transposition of the great arteries: a 2D strain study. Echocardiography 2010;27(10):1256-62.

25. Kalogeropoulos AP, Georgiopoulou VV, Giamouzis G, Pernetz MA, Anadiotis A, McConnell M et al. Myocardial deformation imaging of the systemic right ventricle by two-dimensional strain echocardiography in patients with d-transposition of the great arteries. Hellenic J Cardiol 2009;50(4):275-82.

26. Pettersen E, Helle-Valle T, Edvardsen T, Lindberg H, Smith HJ, Smevik B et al. Contraction pattern of the systemic right ventricle shift from longitudinal to circumferential shortening and absent global ventricular torsion. J Am Coll Cardiol 2007;49(25):2450-6.

27. Davlouros PA, Niwa K, Webb G, Gatzoulis MA. The right ventricle in congenital heart disease. Heart 2006;92 Suppl 1:i27-38.

28. Abdi AH, Luong C, Tsang T, Allan G, Nouranian S, Jue J et al. Automatic Quality Assessment of Echocardiograms Using Convolutional Neural Networks: Feasibility on the Apical Four-Chamber View. IEEE Trans Med Imaging 2017;36(6):1221-1230.

29. Cuypers JA, Eindhoven JA, Slager MA, Opic P, Utens EM, Helbing WA et al. The natural and unnatural history of the Mustard procedure: long-term outcome up to 40 years. Eur Heart J 2014;35(25):1666-74.