Detection and De-occlusion of Text in Images

ISSN: 2277-4629 (Online) | ISSN: 2250-1827 (Print) CPMR-IJT Vol. 2, No. 2, December 2012

www.cpmr.org.in CPMR-IJT: International Journal of Technology 6

Detection and De-occlusion of Text in Images

S. Bhuvaneswari* Dr. T. S. Subashini**

Dr. V. Ramalingam***

ABSTRACTThe proposed work aims to automatically detect thepixel locations corresponding to text regions in animage automatically using CCL and templatematching. These detected pixels are used asinpainting mask to de-occlude them from the imageusing the fast marching inpainting algorithm. Thiswork is done in two steps. The first step detects theregion of text from the image without the usermanually marking it and in the second step the textis de-occluded from the image using the fastmarching inpainting algorithm.

Keywords: Niblack’s algorithm, CCL, Selectioncriteria, Templates, Fast marching algorithm.

I. INTRODUCTIONThe image can be understood as a two dimensionalfunction (x,y) where x and y are spatial coordinates,and the amplitude of f at any pair of coordinates (x,y) iscalled the intensity or gray level of the images at thatpoint [1].

The text data which is present in an image is ofdifferent font styles, sizes, orientation, colors and mostlyagainst a complex background. Text data extraction andmanipulation of objects from digital media is essentialfor understanding editing and retrieving information. In

this work the text region in the image is detected andconcealed using fast marching technique.

The procedure is divided into four steps: detection,localization, extraction, and inpainting. The detection steproughly classifies text regions and non-text regions. Thelocalization step determines the exact boundaries of textstrings. The extraction step filters out background pixelsin the image, so that only the text pixels are left forinpainting.

Inpainting restores a degraded image or video insuch a way that the changes are not apparent to a casualobserver. Photography and film industries are nowadopting this technique to remove artifacts and scratches,to remove undesired objects and text from theforeground or from the background.

Fig. 1 a ) Input image b ) Inpainted image

The common requirement for all image inpaintingalgorithms is that the region to be inpainted should bemanually selected by the user.

*, **, *** Department of Computer Science and Engineering, Annamalai University, Tamilnadu, India



As a first step the user manually selects the portionsof the image that will be restored. This is usually doneas a separate step and involves the use of other imageprocessing tools. Then image restoration is doneautomatically, by filling these regions in with newinformation coming from the surrounding pixels for fromthe whole image.

It can be seen from Fig.1 how the flowers in theoriginal image have been inpainted [9] after manuallyselecting the flower region.

There are many applications of image inpaintingranging from restoration of photographs, films, removalof occlusions such as text, subtitle, logos, stamps,scratches, red eye removal etc.,

The rest of the paper is organized as follows. SectionII describes the related work in this area. Section IIIpresents the key observations and methodology of thiswork. Section IV shows the experimental results.Section V concludes the paper.

II. RELATED WORKThe work in [2] presents an algorithm to inpaint imagesusing cellular neural networks. The results show that analmost blurred image can be recovered with visuallygood effect. In [3] holes were inpainted usingmorphological component analysis designed forseparation of linearly combined texture in a given image.

Authors in [4] used a convolution mask which wasdecided interactively and requires user intervention. Butthis algorithm works only for small regions and cannotinpaint large regions in the image. [5] employs thehorizontal projection and geometric properties for regionsegmentation and selection of text regions.

[6]uses the texture property to identify text usingSVM. [7] analyses the connected component basedalgorithm, and edge based algorithm for text regionextraction. They concluded that the connectedcomponent algorithm is robust and invariant to scale,lighting and orientation compared to the edge basedalgorithm.

The work in [8] applies an algorithm which localizesthe text in images. The detected text are binarized which

can be directly used for text recognition process. Thework in [9] introduces a novel algorithm for digitalinpainting of still images without user intervention. Itremoved target region from digital photographs usingregion filling algorithm.

But the proposed technique does not require anyuser intervention to select the region to be inpainted.Since we intend to de-occlude text from images, wepropose a method to automatically detect text whichwill be used as the inpainting mask during the inpaintingphase.

III. METHODOLOGYThis work is done in two steps. The first step detectsthe region of text from the image without the user manuallymarking it and in the second step the text detected inthe first step is used as a mask for de-occluding it fromthe image using the fast marching inpainting algorithm.

The proposed work is shown in Fig. 2.

Input image

Resizing the image to 400x400

Checking for the light background, if not changebackground as light background

CCL is applied to obtain connected regions

Regions are subjected to selection criteria

New candidate regions obtained and remainingregions are resized in 24x42

Candidate regions compared with templates

Matching candidates are extracted as text

Extracted text is the mask for inpainting

Apply Fast Marching method

Inpainted image

→→

→→

→→

→→

→→

Fig. 2 The methodology of the proposed work



1. The image background and foreground informationis manipulated to have a dark text region againstthe light background.

2. The resultant image is resized to 400 x 400 to reduceunnecessary computations.

3. The resized image is binarized using Niblackalgorithm [10] which takes the local mean andstandard deviation to find out the threshold forbinarizing the image.

4. Connected component labelling (CCL) scans animage and groups its pixels into components basedon pixel connectivity, which means all pixels in aconnected component share similar pixel intensityvalues and are in some way connected with eachother. Once all groups have been determined, eachpixel is labelled with a graylevel or a colouraccording to the component it was assigned to[1].So, CCL is applied to obtain the variousconnected region and the connected regions arebounded using Bounding boxes.

5. Parameters namely criteria namely area, perimeterand the aspect ratio of each region is calculated toeliminate non text region.

6. The remaining candidate text region are now resizedinto 42 x 24.

7. Each resized component is matched with thetemplate to check whether it is a text. A templateof all the text characters (A…Z),(a….z),(0…9)has been created and stored in a template file. Theimages of the character were resized to 42x24 andstored.

8. This further eliminated non text region and theresultant text region will now be considered as theinpainting mask.

9. The image is now inpainted using fast marchingalgorithm [11], which is a technique for producingdistance maps of the points in a region from theboundary of the region. This method combined witha way to paint the points inside the boundary,according to the increasing distance from theboundary of the region.

IV. EXPERIMENTAL RESULTSThe images with simple and complex background arecollected as input. The algorithm is to the images andthe outcomes are shown below:

a. Original b. Binarized image using Niblack’s algorithm

c. Labelled image d. Target regions before applying criteria

e. Target regions after f ) Inpainted imageapplying criteria

Fig.3The text in T shirt has been detected and inpainted

a. Original image b. Binarized image using Niblack’s algorithm



c. Labelled image d. Target regions before applying criteria

e. Target regions after f. Inpainted imageapplying criteria

Fig.4The text in a Poster is detected and inpainted

a. Original image b. Binarized image using Niblack’s algorithm

c. Labeled image d. Target regions before applying criteria

e. Target regions after f ) Inpainted image applying criteria

Fig.5The text in Car name plate detected

and inpainted regions

V. CONCLUSIONIn this paper, we have presented a method to detecttext automatically in colour images. The detected textregion is the target region for inpainting.

Niblack’s algorithm was applied to binarize theimage, CCL and template matching was carried out todetect the text. The text detected was de-occluded usingfast marching algorithm. Experimental results havedemonstrated that the proposed method can beeffectively used to detect the text automatically. One of the limitations of this work is that the proposed workfails to detect joined running text, and fancy text fonts.

VI. REFERENCES[1] R.C.Gonzalaz and R.E.Woods, Digital Image

Processing, 2nd ed., Pearson Education, 2002.

[2] M.J.Fadiii,J.L.Starck and F.Murtagh,“Inpainting and Zooming using sparseRepresentation”, The computer Journal, pp.64-79, 2009.

[3] A.C.Kokaram, R.D.Morris, W.J.Fitzgerald,and P.J.W.Rayner, “Interpolation of missing datain image sequences”. IEEE Transactions onImage Processing, Vol. 11, No.4, pp.1509-1519, 1995.



[4] A.Criminisi, P.Perez, K.Toyama, “Region Fillingand object removal by exemplar based imageinpainting”, IEEE transactions on imageprocessing, Vol.13, No.9, pp.1-7, 2004.

[5] RodolfoP.DosSantos, S.Gabriela, Clemente,TsangIng Ren, George D.C.Calvalcanti, “TextLine segmentation based on morphology andhistogram projection”, IEEE Proceedings onthe 10th International conference on documentanalysis and recognition (ICDAR), pp.651-655,2009.

[6] Qixiang Ye,Wen Gao,WiquiangWang,WeiZeng, “ A robust text detectionalgorithm in images and video frames”, IEEEProceedings on the International conference oninformation, communication and signalprocessing (ICICS), Vol.2, pp.802-806, 2003.

[7] J.Sushma, M.Padmaja, “Text Detection in colorimages”, IEEE Proceedings on the Internationalconference on intelligent agent & multi agentsystems (IAMA), pp.1-6, 2009.

[8] Udaymodha, Preethidave, “image inpainting -Automatic Detection and Removal of Text from

images”, International journal of engineeringresearch and applications (IJERA), Vol.2,No.2, pp-930-932, 2012.

[9] S.Bhuvaneshwari, T.S.Subashini,S.Soundharya, V.Ramalingam, “A novel and fastexemplar based approach for filling portions inan image”, IEEE Proceedings on theInternational conference on recent trends ininformation technology (ICRTIT), pp.91-96,2012.

[10] Graham Leedham, Chen Yan Kalyan Takru, JoieHadi Nata Tan and Li Mian, “Comparison ofSome Thresholding algorithms for text/Background Segmentation in difficult documentimages”, IEEE Proceedings on the Seventhinternational Conference on Document Analysisand Recognition (ICDAR), pp.859-864, 2003.

[11] Alexandru Telea,”An image InpaintingTechnique Based on the Fast MarchingAlgorithm”, Journal of Graphics tools,Vol.9,No.1, pp.25-38, 2004.

Documents

Detection and De-occlusion of Text in Images