Journal of Electrocardiology 51 (2018) 707–713

Contents lists available at ScienceDirect

Journal of Electrocardiology

j ourna l homepage: www. jecgon l ine .com

Image digitization of discontinuous and degraded electrocardiogrampaper records using an entropy-based bit plane slicing algorithm

Rupali Patil a,⁎, Ramesh Karandikar b

a Department of Electronics and Telecommunication, Rajiv Gandhi Institute of Technology, Mumbai 400053, Indiab Deparment of Electronics and Telecommunications, K. J. Somaiya College of Engineering, Mumbai 400077, India

⁎ Corresponding author.E-mail address: rupalipatil@somaiya.edu (R. Patil).

https://doi.org/10.1016/j.jelectrocard.2018.05.0030022-0736/© 2018 Elsevier Inc. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Available online xxxx

Background: Electrocardiograms (ECGs) are routinely recorded and stored in a variety of paper or scanned image format. Current ECG recording machines record ECG on graph paper, also it provides digitized ECG signal alongwith automated cardiovascular diagnosis (CVD). However, such recording machines cannot analyse preservedpaper ECG records as it requires input in terms of digitized signal. Therefore, it is important to extract ECG signalfrom these preserved paper ECG records using digitization method. There are different paper degradations thatadversely affect digitization process. The purpose of this work is to perform an image enhancement and digitiza-tion of the degraded ECG images to extract continuous ECG signal.Methods: In this paper, we propose entropy-based bit plane slicing (EBPS) algorithm in which pre-processing isdone using dominant color detection and local bit plane slicing. Maximum entropy based adaptive bit plane se-lection is applied to the pre-processed image. Discontinuous ECG correction (DECGC) is then done to producecontinuous ECG signal.Results: The algorithm is tested on 836 different degraded paper ECG records obtained from various diagnosticcenters. After analysis with 101 known ground truth ECG signals the accuracy, sensitivity, specificity and overallF-measure of ECG is 99.42%, 99.69%, 99.81% and 99.26% respectively. The RMS error and correlation between theextracted digitized signal and ground truth for 101 cases is 0.040 and 99.89% respectively.Conclusions: The EBPSmethod is able to remove all types of degradation in paper ECG records to generate a uniformdigitized signal. Instead of manual measurement and prediction from archived paper ECG records, automated pre-diction (using already existing cardiovascular diagnosis software) is possiblewith the help of extracted digitized sig-nal obtained using proposed digitization method, which will also help retrospective cardiovascular analysis.

© 2018 Elsevier Inc. All rights reserved.

Keywords:Entropy-based bit plane slicingDominant color extractionDegraded ECGDiscontinuous ECG

Introduction

Electrocardiogram (ECG) is commonly recorded for analysis of car-diac problems. In developing countries [1], roughly about 90% hospitalsor clinical departments routinely record ECG on a variety of graph pa-pers and these records are preserved by either patient or clinical depart-ment. Such preserved ECG records are degraded [2] with various typesof noise like signal discontinuity due to ink evaporation, blurring andfolding of paper ECG record. This paper majorly focuses on preservedor older resting ECGs. Nowadays printed ECG graph paper is scannedand stored as an image. The problems with storage in the form of thescanned image are, it requires large storage space, transmission requiresmore bandwidth and time, the addition of noise during scanning (suchas marginal noise) [3,4]. Though previous research focused on digitiza-tion of scanned image of the printed signal [5,6], nowadays image canbe immediately obtained with a cell phone [7]. However such cell

phone images also have problems like no uniform illumination, lowcontrast due to lack of sufficient or controllable lightning, and lack of ad-justable camera exposure time and aperture size. All these problemsmake the scanned image as well as cell phone captured image difficultfor diagnosis. The focus of this work is on handling various degradationsoccurring in older resting ECGs and extraction of clear and continuousECG signal trace for better analysis and interpretation.

Latest ECG recordingmachines stores ECG recording in terms of dig-itized ECG signal along with automated cardiovascular diagnosis (CVD)[8,9]. However, such recordingmachines, which has automated diagno-sis software cannot analyse preserved paper ECG records as it requiresinput in terms of one-dimensional time series digital signal (vector)and not as paper ECG record or as scanned ECG image. Hence to developan application for digitization and image enhancement (noise removal)of preserved (degraded) paper ECG records and extracting signal fromunstructured ECG graph paper records [10,11] is the immediate need.These digitized signals may be useful for retrospective cardiovascularanalysis [12]. In the long run, it is important to conserve the docu-mented ECG graph paper records in a uniform format and in single

708 R. Patil, R. Karandikar / Journal of Electrocardiology 51 (2018) 707–713

database [1] for quick retrieval in datamining tomaximizing research inthe area of cardiac electrophysiology.

Methods

In the proposed entropy-based bit plane slicing (EBPS) algorithm,first degraded ECG graph paper records are scanned at 200 dpi (dotsper inch) [13,14]and stored in png format. The size of the image isthen measured in terms of spatial resolution (pixels) in MATLAB andeach side of the image is cropped down in order to remove any borderpresent in an ECG signal (cropping size of 40 pixels from each side).

Dominant color extraction [15] is carried out by separating colorplanes and considering each R, G and B plane separately as a greyscale image. The average values from each of these grey scale imagesare extracted after neglecting pixels in vicinity ±5 pixels). Sampleaverage value of red color plane is computed using formula shownbelow:

AVGR ¼X

m¼i

X

n¼ j

I m;nð Þ= i � jð Þ ð1Þ

∀Im;n∉AVGR � 5 pixel

where,

AVGR = average grey value of red color planei = all number of rowsj = all number of columnsI (m,n) = each pixel value at mth and nth instance of the loop

Fig. 1. Last three bit planes out of extracted eight bit planes (BP) from scanned paper ECG imageand non-degradedECG's).The required information (i.e. ECG signal trace)was available in 7th andecide which bit plane has maximum information about ECG trace.

Similarly, sample averages are calculated for B and G color planes aswell. The dominant color is determined using these three R G B valuesand the adaptive threshold is then applied. The threshold is set as perthe average grey value of the image as shown by Fig. 6B. After dominantcolor determination, grey plane, red plane, green plane and blue plane(RGB planes) are separated from the image. The selected grey planeimage is then contrast stretched to the limits from 0 to 255. After contraststretching all the 8 different bit planes are extracted [16] from the selectedplane imageas shown in Supplementary Fig. S2.Mostly the required infor-mation (i.e. ECG signal trace) is available in 7th and8thbit plane, as shownin Fig. 1. Out of eight extracted planes, correct plane (which has only ECGsignal trace) is selected usingmaximumentropymethodwhich separatesECG signal trace from background grid as per the following formula,

Im x; yð Þ ¼ IN x; yð Þ;∀IN x; yð Þ∈R ð2Þ

where,

R- range of values lying between 2m−1 and 2m

m- current bit plane under computationx- range from 1 to my- range from 1 to n and (m, n) is size of image and (x, y) is currentpixel under computation and if Im(x, y) exists then

IN−1 x; yð Þ ¼ IN x; yð Þ−2m−1 ð3Þ

where,

m = bit planeIN = input image for mth bit plane extraction

(each row corresponds to a bit plane and column shows different inputs such as degradedd8thbit plane.Hence the adaptive selection of bit plane is needed for further processing to

709R. Patil, R. Karandikar / Journal of Electrocardiology 51 (2018) 707–713

IN−1 =modified input the image for (m− 1)th bit plane extraction.After bit plane slicing corresponding entropy (H) is calculated asfollows,

H ¼XL−1

i¼0

−pi log2 pið Þ ð4Þ

where L is a number of grey levels (i = 0, 1, L1) and pi is the probabilityof occurrence of grey level i in the image. As per the maximum entropyvalue, the threshold is set. Depending upon the image illumination thebit plane to be selected varies and hence adaptive bit plane selectionis carried out using minimum entropy that crosses the threshold of0.0001. Once the current minimum value of entropy is found out to bebelow the threshold, the minimum value is replaced by a maximumvalue and new minimum value of entropy is calculated, this process isrepeated until sufficient information contents i.e. 0.1 ≤ H ≤ 0.4 is foundout.

The EBPS algorithm is tested for grey plane as well as RGB planesof the scanned paper ECG record which has different color back-ground grids used by different ECG recording machines. As ECG sig-nal (required information) is always recorded (printed) with blackcolor irrespective of different background colors (red, blue, pinketc.), only grey plane of scanned paper ECG record is selected forEBPS processing. The grey and RGB plane, tested results are shownin Fig. 2, which shows that, grey plane with EBPS processing canclearly separates required ECG signal from different backgroundgrid colors, compared to RGB planes. The entropy-based algorithmgives better results because the removed background has uniformgrey value leading to zero entropy (i.e. no information) and alsomost of the background is removed in pre-processing steps, while asmall portion of ECG signal in an image corresponds to very small en-tropy value (Typically ranging between 0.1 and 0.4). Once the

Fig. 2. Grey plane selection irrespective of different background color used by different manureferred to the web version of this article.)

correct bit plane from grey plane of scanned image is selected, it iscopied in all remaining 8-bit planes to achieve a black and whiteimage with grey-scale values ranging from 0 to 255.

The proposed EBPS algorithm is tested on images with various deg-radations. Degradation is the deviation of black ink level on graph paper.This change in black ink level is represented by degradation parameteras follows

DP ¼ 1m � n∑

1m∑

1n Ic−Ibð Þ ð5Þ

where, DP = degradation parameter Ic = degraded image black inklevel Ib = original black ink level (Ib = 0, if ground truth not available)andm, n are rows and columns of scanned ECG image. Fig. 3 shows pro-posed algorithm can handle different types of degradation and can ex-tract required ECG signal. The resultant image is then verticallyscanned, where breakage or discontinuities are detected. Correction ofbreakage due to the steep slope is doneusing discontinuous ECG correc-tion (DECGC) method. Due to steep slopes in ECG, driving speed of thepen attached to ECG recorder is also high. This leads to low contrasttrace on the ECGpaper as shown in Fig. 4(A–E). Hence during extractionof R-peaks, ECG signal breaks most of the times, giving unwanted dis-continuities in image processing. Fig. 4B and C shows two types ofsteep-slope discontinuities. It is again observed that all types of degra-dation lead to the common problem of the discontinuous signal traceas shown in Fig. 4(F–J).

Fig. 4D shows when the slope is positive, the signal pixel (SP) fromcolumn 1 (C1) i.e. previous vertical scan line is below the signal pixelfrom the next pixel scan line i.e. column 2 (C2). If the distance betweenthese two pixels is more than one pixel then the current column is filledfrom C1-SP (signal pixel SP from C1 column) to C2-SP point (Fig. 4E). Incase 2 as shown in Fig. 4E, C1-SP is above C2-SP leading to a large neg-ative slope again. Here delta Δ is calculated as (y2− y1) and if this Δ is

facturers. (For interpretation of the references to color in this figure legend, the reader is

Fig. 3. Types of degradations (left) and its effect on bit plane extraction algorithm (right)(A) variable background color, (B) ink evaporation effect, (C) folding and blurring effect,(D) marginal noise, (E) ink spreading due to steep slope, (F) non-uniform illuminationdue to cell phone camera capture.

Fig. 4. (A–E) Steep slope correction, (F–J) discontinuous ECG correction, (K) continuoustrace plot using fit function.

710 R. Patil, R. Karandikar / Journal of Electrocardiology 51 (2018) 707–713

greater than one pixel then column1 is filledwith signal pixels up to y2.The next part of the algorithm is used to connect the missing signalpixels in the entire column (Fig. 4I and J). For this particular case asshown in Fig. 4J the signal pixel is found at C1 and next signal pixel is di-rectly found in C3 or in general column n (Cn), where the signal pixel ofC1 is at higher value than signal pixel found at column n, this indicatestheremust be a negative slope pixel between C1 and Cn. Here we calcu-lated a number ofmissing signal pixel columns using following formula:

x ¼ N−1ð Þ–1 ð6Þ

and Δ is same as earlier two cases. Number of pixels to be filled per col-umn is calculated by the formula.

J ¼ Δ=x ð7Þ

Finally, (J + 1) pixels are filled between signal pixels at column 1 tosignal pixel at column n. A similar approach is followed for Fig. 4I exceptit is for a positive slope.

Once the final image is obtained, each column is assigned a singlevalue called center point. The central point is determined by calculatingminima andmaxima and settingmean ofminima andmaxima as a cen-tral point to one for that particular column. Once all the center pointsare determined for each column they are connected using a four-pointspline fit to obtain smooth ECG trace as shown in Fig. 4K. The image

Fig. 5. (A) Performance parameter. (B) Threshold value of dominant color detection as a function of average grey value. (C)Degraded ECG image, (D) discontinuous ECG signal trace,(E) DECGC, (F) Ground truth. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Table 1Level of degradation for each type of degradation and RMS error between the reconstructed signal and ground truth.

Type of degradation No of samples (N = 101) RMS error Degradation parameter Correlation (%)

Variable background color 32 0.011 0.0357 99.96Ink evaporation effect 24 0.029 0.2443 99.92Folding effect 07 0.14 0.1825 99.78Marginal noise 03 0.02 0.0264 99.90Ink fading due to steep slope 31 0.023 0.1416 99.90Non uniform illumination 04 0.021 0.0337 99.93

Table 2Comparison of existing paper ECG digitization methods with proposed method.

Method No. ofsamples

Accuracy(%)

RMS error(%)

Correlation(%)

Random transform [20] 10 95 – –Contrast enhancement [1] filtering 5 99 98Neighbourhood and medianapproach [21]

5 – 3

Anchor point setting [19] 169 – 16.8 95Global thresholding [13] 114 – – –Morphological feature extraction[22]

25 97

Grid filtering [23] 112 99Binarization and thinning [24] 50 98Signal contour extraction [25] 30 – 12 –Grid detection and OCR [26] 24 – – 100Proposed method 101 99 4 99

Bold defines or highlights results obtained by proposed method when compared withmethods reported in literature.

711R. Patil, R. Karandikar / Journal of Electrocardiology 51 (2018) 707–713

with smooth ECG signal trace is then scanned vertically to extract timeseries digitized ECG signal. The extra pixels or samples are removedusing morphological thinning and finally, the digitized signal is plottedin MATLAB as final ECG signal graph as shown in Fig. 6E. The extracteddigitized signal is compared with ground truths for validation of a pro-posed algorithm.

Results

In all 836 ECG signals are tested. For the validation of the algorithmdigitized ECG (N=101) is comparedwith ground truth of the signal ob-tained from direct digital data machines. They are plotted in same timescale as shown in Fig. 6E and F. Fig. 5A shows the accuracy, sensitivityand, specificity [17] of this method as 99.42%,99.69% and 99.81% respec-tively. The overall F-measure of ECG is 99.26%.Thereafter the algorithmis applied to the images (N = 735) for which ground truths are notavailable and visual inspection of the digitized output is done by twocardiologists. Although in the present work we have shown entire ECGextracted as the output, it is always useful to extract baseline sample be-tween the beat as reported in our earlier work [18].Fig. 5B shows aver-age grey values is used as threshold point for dominant color detection.As shown in Fig. 2, grey plane comparatively gives better results irre-spective of different background color paper used by ECG recorderfrom different manufacturers. Table 1 shows (degradation parameter)that degradation is maximum for Ink evaporation effect or blurring ef-fect, which is the very common case for older resting ECGs. Root meansquare (RMS) error and correlation [19] is calculated to evaluate howclose are the digitized ECG pixel values obtained from the algorithm tothe actual ECG (Ground truth).The RMS error and correlation for 101cases is 0.040 and 99.89% respectively. When compared with existingpaper ECG digitization methods, a proposed method performs betterin terms of accuracy, RMS error, and correlation, as shown in Table 2.

Discussion

The results revealed that the signal obtained through proposed dig-itization process retained maximum information from the degradedpaper ECG records. Compared with currently available techniquesmethods [10,12, 26] the key feature of the proposed technique is thatit is able to reconstruct the original ECG signal despite the degradationsthat usually occur in older resting paper ECG records. The paper ECGdegradations like variable background color, ink evaporation effect,folding and blurring effect, marginal noise, ink fading due to the steepslope, non-uniform illumination due to cell phone camera capture arerobustly handled using EBPS and DECGC method for extraction of

Fig. 6. Need for DECG. (A) Original ECG scan record. (B) Eighth-bit plane output. (C) Direct Fitting without DECG applied. (D) Spline fitting applied after DECG. (For interpretation of thereferences to color in this figure, the reader is referred to the web version of this article.)

712 R. Patil, R. Karandikar / Journal of Electrocardiology 51 (2018) 707–713

continuous digital ECG signal. The dominant color is determined using,R G B values and the adaptive threshold is then applied to ECG signals.The threshold is set as per the average grey value obtained from thedominant color of the scanned image. Depending upon the image illu-mination the bit plane to be selected varies and hence adaptive bitplane selectionwas carried out using entropy value. Due to degradationthere exist a common problem of the discontinuous trace, hence onlyEBPS algorithm is not sufficient enough, we need DECGC method incombination with EBPS. Fig. 6 shows need for DECG. When the directlinear fitting is applied to the 8th-bitplane (blue line in Fig. 6C), the re-covered ECG had sharp edges which are not possible with ECG signals.When direct spline fitting is applied on DECG (red line in Fig. 6C), in-stead of applying it after DECG step (Fig. 6D), the output waveform(red line Fig. 6C) is having unwanted secondary peaks generated.Hence it is important in some cases that DECG must be applied beforefinal spline fit. The proposed algorithm results in an error when thelead names e.g. V1, V2 etc. overlapped with ECG signal trace. It alsofails for large fold occurred during scanning paper ECG records. The pro-posed algorithm reliably converts thermal paper records but not limitedto only thermal papers.

The algorithm can work for any paper printouts to extract digital in-formation, even if they are faded or recently printed. One of the poten-tial applications of the presented work is to extract bio-medical or anyother signal printed on old paper records.

Conclusion

The preserved ECG paper records degrade due tomany reasons suchas time-based ink evaporation, improper handling, folding etc. Due tothis paper degradation, there is a need for image enhancement

algorithm after scanning these records to extract useful informationout of it for automated cardiac retrospective analysis. We have devel-oped image enhancement algorithm and tested over 836 ECG paper re-cords. The accuracy of this method is 99.42% and sensitivity, specificityis found out to be 99.69% and 99.81% respectively. The RMS error andcorrelation between the extracted digitized signal and ground truthfor 101 cases is 0.040 and 99.89% respectively. All the possible degrada-tions in paper ECG records are successfully handled and digitized ECGsignal is extracted using EBPS algorithm. We expect this technique towork not only on ECG paper records, but on all paper-based outputmachines.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jelectrocard.2018.05.003.

References

[1] Kumar V, Sharma J, Ayub S, Saini J. Extracting samples as text from ECG strips forECG analysis purpose. Computational intelligence and communication networks(CICN), 2012 fourth international conference on. IEEE; 2012. p. 317–21.

[2] Su B, Lu S, Tan CL. Robust document image binarization technique for degraded doc-ument images. IEEE Trans Image Process 2013;22(4):1408–17.

[3] Farahmand A, Sarrafzadeh A, Shanbehzadeh J. Document image noises and removalmethods. Proceedings of the international multiconference of engineers and com-puter scientists, 1. ; 2013.

[4] Farahmand A, Sarrafzadeh H, Shanbehzadeh J. Noise removal and binarization ofscanned document images using clustering of features. Internationalmulticonference of engineers and computer scientists; 2017. p. 410–4.

[5] G. e Silva A, de Oliveira HM, Lins RD. Converting ECG and other paper legated bio-medical maps into digital signals. International workshop on graphics recognition.Springer; 2007. p. 21–8.

713R. Patil, R. Karandikar / Journal of Electrocardiology 51 (2018) 707–713

[6] Patil R, Karandikar R. Digital signal preservation approaches of archived biomedicalpaper records — a review. Wireless networks and embedded systems (WECON),2016 5th international conference on. IEEE; 2016. p. 1–4.

[7] Lozano-Fernández F, Mora-Jiménez I, Sanromán-Junquera M, Muñoz-Romero S,García-Alberola A, Rojo-Alvarez JL. Auto-cropping of phone camera color images tosegment cardiac signals in ECG printouts. Computing in cardiology conference(CinC), 2016. IEEE; 2016. p. 421–4.

[8] Holt TA, Dalton A, Marshall T, Fay M, Qureshi N, Kirkpatrick S, et al. Automated soft-ware system to promote anticoagulation and reduce stroke risk. Stroke 2017;48(3):787–90.

[9] Santos HS, Dutra OO, Moreira EM, Ferreira LH. Android framework for automatic di-agnosis of heart diseases. Medical measurements and applications (MeMeA), 2017IEEE international symposium on. IEEE; 2017. p. 216–21.

[10] Sanromán-Junquera M, Mora-Jiménez I, Caamaño AJ, Almendral J, Atienza F, CastillaL, et al. Digital recovery of biomedical signals from binary images. Signal Process2012;92(1):43–53.

[11] Chatterjee H, Gupta R, Mitra M. A statistical approach for determination of timeplane features from digitized ECG. Comput Biol Med 2011;41(5):278–84.

[12] Waits GS, Soliman EZ. Digitizing paper electrocardiograms: status and challenges. JElectrocardiol 2017;50(1):123–30.

[13] Lawson W, Wagner G, Startt-Selvester R, Ybarra G. Newmethod for digitization andcomputerized analysis of paper recordings of standard 12-lead electrocardiograms.Computers in cardiology 1995. IEEE; 1995. p. 41–4.

[14] Patil R, Karandikar R. Digitization of documented signals using vertical scanning. Mi-crowave, optical and communication engineering (ICMOCE), 2015 internationalconference on. IEEE; 2015. p. 239–42.

[15] Choroś K. Automatic playing field detection and dominant color extraction in sportsvideo shots of different view types. Multimedia and network information systems.Springer; 2017. p. 39–48.

[16] Thepade S, Das R, Ghosh S. A novel feature extraction technique using binarization ofbit planes for content based image classification. J Eng I 2014:13.

[17] De Chazal P, O'Dwyer M, Reilly RB. Automatic classification of heartbeats using ECGmorphology and heartbeat interval features. IEEE Trans Biomed Eng 2004;51(7):1196–206.

[18] R. Patil, R. Karandikar, Robust algorithm for digitization of degraded electrocardio-gram paper records., ICTACT J Commun Technol 8 (3).

[19] Badilini F, Erdem T, Zareba W, Moss AJ. ECGScan: a method for conversion of paperelectrocardiographic printouts to digital electrocardiographic files. J Electrocardiol2005;38(4):310–8.

[20] Swamy P, Jayaraman S, ChandraMG. An improvedmethod for digital time series sig-nal generation from scanned ECG records. Bioinformatics and biomedical technology(ICBBT), 2010 international conference on. IEEE; 2010. p. 400–3.

[21] Chebil J, Al-Nabulsi J, Al-Maitah M. A novel method for digitizing standard ECG pa-pers. Computer and communication engineering, 2008. ICCCE 2008. Internationalconference on. IEEE; 2008. p. 1308–12.

[22] Damodaran V, Jayaraman S, Poonguzhali S. A novel method to extract ECGmorphol-ogy from scanned ECG records. Defense science research conference and expo(DSR), 2011. IEEE; 2011. p. 1–4.

[23] Bhullar HK, Fothergill JC, Goddard WP, de Bono DP. Automated measurement of qtinterval dispersion from hard-copy ECGs. J Electrocardiol 1993;26(4):321–31.

[24] Mitra S, Mitra M, Chaudhuri BB. Generation of digital time database from paper ECGrecords and Fourier transform-based analysis for disease identification. Comput BiolMed 2004;34(7):551–60.

[25] Karsikas M, Huikuri H, Perkiömäki JS, Lehtola L, Seppänen T. Influence of paper elec-trocardiogram digitizing on t wave and qrs complex morphology parameters. AnnNoninvasive Electrocardiol 2007;12(4):282–90.

[26] Ravichandran L, Harless C, Shah AJ,Wick CA, Mcclellan JH, Tridandapani S. Novel toolfor complete digitization of paper electrocardiography data. IEEE J Translational EngHealth Med 2013;1 1800107–1800107.