Development of a Container Identification Mark Recognition System

Shintaro Kumano,1 Kazumasa Miyamoto,1 Mitsuaki Tamagawa,1 Hiroaki Ikeda,2 and Koji Kan3

1Takasago Research and Development Center, Mitsubishi Heavy Industries, Ltd., Takasago, 676-8686 Japan

2Hiroshima Research and Development Center, Mitsubishi Heavy Industries, Ltd., Mihara, 729-0393 Japan

3Kobe Shipyard and Machinery Works, Mitsubishi Heavy Industries, Ltd., Kobe, 652-8585 Japan

SUMMARY

This paper reports the development of a containeridentification mark or number recognition system designedfor application to a container terminal. The recognitionsystem recognizes the number or mark on the back surfaceof a container in an outdoor environment by photographingthe mark by a camera system installed on the container gate.Containers in many cases differ in color as well as in layoutdepending on their owners; the layouts commonly containone to four horizontal columns or rows of writing or morerarely vertical rows of writing. The container number rec-ognition system is constructed from an illumination inten-sity sensor and illumination system for handling theoutdoor illumination changes, a shutter speed control de-vice, and devices such as filters for handling various con-tainer colors. In addition, the proposed system uses acharacter recognition scheme based on a dynamic designmethod for recognizing differing character string layouts incontainer marks or numbers. Field tests have been con-ducted to obtain a recognition rate of 92.8% for all data, arecognition rate of 97.9% for effective or appropriate dataexcluding data outside the field of vision, and an averagerecognition speed of less than 2 seconds. © 2004 WileyPeriodicals, Inc. Electron Comm Jpn Pt 2, 87(12): 38–50,2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.20134

Key words: character recognition; low qualitycharacters; weighted directional index histograms; dy-namic designing; container number reading.

1. Introduction

In recent years, the amount of containers transportedhas increased with the trends of increasing sizes of shipsand decreasing transportation costs and charges. In addi-tion, with increasing participation of developing countriesin container services, competition among individual con-tainer terminals has also increased as well as the urgency ofthe need to meet the new requirements of equipment, per-sonnel, and the like. Constructing comprehensive auto-mated and simplified systems is needed to increase theefficiency and productivity of systems established to oper-ate 24 hours daily for 365 days in such circumstances [1].

Targets of improvement for container terminals en-compass a wide range that includes unloading containers,transporting containers inside a yard, and managing con-tainer contents. Container numbers are commonly used inthese systems and automatic recognition techniques forcontainer numbers are considered to constitute the nucleusof automated systems for container terminals. As the firststep in realizing the container number recognition tech-niques based on such analyses, container number recogni-tion devices for container gates have been developed. A

© 2004 Wiley Periodicals, Inc.

Electronics and Communications in Japan, Part 2, Vol. 87, No. 12, 2004Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J84-D-II, No. 6, June 2001, pp. 1073–1083

38

container number recognition device can automate as wellas simplify and shorten manual work in recognizing con-tainer numbers, which gate clerks have performed untilnow.

The conventional container automatic recognitionmethods are classified broadly into methods of readingnumbers optically [3–5] and methods using ID tags [2].Although the methods using ID tags are superior from thepoint of view of recognition accuracy, they cannot bewidely used due to the cost associated with ID tags installedon all containers. On the other hand, recognition methodsbased on optical imaging of container numbers cannotachieve an accuracy commensurate with that of the ID tagmethods due to such effects as contamination or scratchingof container numbers. However, optical recognition meth-ods can be implemented by improving automation speedand depending on human judgment in unreadable cases.

For the recognition of English characters and num-bers in outdoor environments, methods of recognizing ve-hicle numbers and the like in addition to container numbershave been researched and developed in Japan and abroad[9–11]. A survey of character recognition techniques andthe character recognition contest held under the sponsor-ship of the Postal Ministry summarized the concepts ofeffective methods [7]. The authors of this paper participatedin the character recognition contest mentioned above andhave developed techniques for recognizing low-qualitycharacters in Japanese and Singaporean vehicle numbers[12–14].

However, in realizing container number recognitiondevices, not only recognizing single characters but alsostudying the steps preceding recognition are important. Forexample, many problems that must be resolved beforeapplying these devices to character recognition processingexist, such as photographing techniques for obtaining bettercontrast of characters in various color combinations, suchas red characters on a white background, or white characterson a brown background, under changing daylight condi-tions due to time or weather in outdoor environments,techniques for extracting character strings to be recognizedsuch as one to four horizontal rows of writing or verticalwriting in character string layouts from characters on ob-jects other than recognition targets.

The authors have previously developed a containernumber recognition device based on combining high-accu-racy recognition of low-quality characters by software andclear photography by hardware. This paper explains thesoftware configuration and the fundamentals of the devicein Section 2, and image processing and software processingof character recognition in Section 3. It also presents theresults of testing the device in a real container yard, andevaluates the efficacy of the device in Section 4. Section 5presents conclusions and future tasks.

2. System Configuration

2.1. Container number

A container number (Fig. 1) usually consists of afour-character owner code (English) and six numbers,along with a single check digit (frame number). The fourthof the four English characters of the owner code is “U” andthe remaining three characters are assigned to an owner ofeach country. There are over 1000 owner codes; the Inter-national Container Bureau registers and manages them [6].Standards ISO6346 and JIS Z 1615 specify the sizes andcontent of the characters.

Although the container number is written on the top,side, front, and back surfaces of a container, the proposeddevice aims to recognize container numbers written on theback surface of a container. This is done for the followingreasons.

(1) The results of a field survey reveal that the char-acter part deteriorates most severely on the upper surfacedue to soiling and scratches caused by piling up of contain-ers, and deteriorates least on the back surface of a container.

(2) In photographing a container at a container gate,it is easier to photograph a character string within the fieldof vision on the back surface of a container than that on theside, due to camera and object relations.

(3) There are convex and concave surfaces (corruga-tions) on the side surfaces of a container, and the brightnessof a character string on a convex surface differs greatly fromthat of a character sequence on a concave surface.

The camera device is assumed to be installed on acontainer gate and used under outdoor conditions. In addi-tion, the device has a configuration that allows the photo-graphing of moving containers and of containers differingin length (20 feet, 40 feet) with the same camera. Figure 2shows the camera system layout. The proposed techniques

Fig. 1. Examples of container ID mark.

39

have been evaluated by recognizing containers with hori-zontal writing, which are more frequently encountered, andin some cases, parts of the vertically written containers wereoutside the field of vision of the camera. The latter casesmay be handled by using a high resolution camera.

2.2. Camera system

The functions needed to obtain good photographicimages using the camera layout shown in Fig. 2 are pre-sented below (Fig. 3 shows a configuration having thesefunctions).

(1) A camera trigger sensor detecting whether a mov-ing container has passed a specific location and a cameracapable of photographing by an external trigger

(2) A light intensity sensor for measuring the bright-ness in outdoor environments

(3) A camera that takes the entirety of a containernumber string into its field of vision, having the capabilityof decomposing an image of a container character width (10mm) within its set field of vision

(4) An illuminating device for use when outdoorenvironments become dark (below 1000 lux)

(5) A shutter speed control device that controls thebrightness of an image corresponding to changes in outdoorlight intensity

(6) A filter that is optimal for handling combinationsof container colors and character colors handled by a con-tainer terminal

(7) A photographic control device for controlling theentire camera system sequentially

2.3. Evaluations of the device

The character area on the back surface of a containermeasures 1.6 m × 1.2 m. The character width of a containernumber is 10 mm at the minimum, the character interval is5 mm, and a resolution of 2.5 mm pixel is required in orderto quantize 5 mm with a minimum of 2 pixels. Based onthis fact, a camera with a VGA monochrome capabilityhaving a size of 640 × 480 pixels is used. We selected theIK-542 (Toshiba), which is capable of controlling 8.94 µsof marking up to 10 µs to 16 ms by external signals. Amonochrome camera and optimal filter are combined inorder to reduce the cost of a color camera.

An optimal filter has been selected from the point ofview of maximizing the contrast of camera images forcombinations of actually existing container colors and char-acter colors. Figure 4 shows evaluation results of G and Rfilters among the luminescence evaluation results obtained

Fig. 2. Camera system layout.

Fig. 3. Container ID mark recognition system. Fig. 4. Filtering results for various color combinations.

40

by using RGB filters for combinations of various typicalcontainer colors. The bar graphs in the figure show thedifferences in luminescence between the characters andbackground parts; the contrast is expressed by the length.The above color combinations can handle colors other thanbrown-black and red-black with a G filter, and combina-tions of brown-black and red-black can achieve the contrast(32 luminescence grades) targeted by an R filter. This studyuses a G filter, considering the frequencies of occurrence ofcolors. A future study will process multiple images sequen-tially by using combinations of two filters and two cameras.

Table 1 shows the conditions of the camera systemestablished in this study.

3. Container Mark Recognition Scheme

3.1. Overall processing configuration

Figure 5 shows an example of a container identifica-tion mark. As can be seen from this figure, in addition tocharacters that are recognition objects, a locking bar (me-tallic), images of convex and concave parts of the back

surface, and characters associated with the country nameand weight coexist in the image. Furthermore, the characterstrings are not limited to specific locations and the writingpatterns are also varied, such as one to four horizontal rows,and one vertical row. There are also cases in which charac-ters are bright or dark against the background, and thecharacter sizes and intervals vary.

This study also investigates the process up to deter-mining the character string ranges as part of the characterrecognition method. Although the general process up to thedetermination of the character string range in the past hasconsisted of the binarization and projection of an imagewithout character recognition, this study infers informationon the sizes and layouts of characters, and information onthe brightness relationship between characters and back-ground by detecting the character “U,” which is necessarily

Table 1. Image capture conditions

Vision field range 1.6 m (horizontal) × 1.2 m(vertical)

Resolution strength 2.5 mm (horizontal) × 2.5mm (vertical)

Number of pixels 640 × 480

Camera lens focusing dis-tance

35 mm

Photographing distance 8.167 m

Constriction F4 fixed

Lens resolution strength over 50% at 25 lines/mm

Photographed field depth 7.12–9.58 m

allowed up to the fadingamount of 1/2 pixel (5 µm)with constriction of F4

Shutter time due to imageflow of moving object

less than 1/326 second

flow of 1 pixel at movingrate of 30 km/h allowed

Filter G

Illuminating device average light intensity of1400 lux (light with lessthan 1000 lux ofenvironmental illumination)

Camera trigger sensor less than 2 m of distance setbefore target

Fig. 5. Variations of container ID mark.

41

included in a character string. In addition, even if “U” is notfound due to deterioration or cutting of a container, thelocation of a character string is estimated by a conventionalmethod based on correlations [12] and the characters areinterpreted as the container identification number by thesubsequent character recognition process.

Since the conventional assumption of “one label, onecharacter” does not necessarily hold even in the recognitionprocess after the extraction of a character string, a methodof recognizing characters and cut characters simultaneouslyby dynamic design methods [15, 16, 18], involving addressrecognition and a subsequent English recognition system,is used. Furthermore, since the recognition result for eachcharacter is influenced by the quality of the binarizedimage, the scheme (multiplexed binary image recognitionmethod) of choosing the most likely characters (based onthe values obtained by subtracting a constant from thedistances from the dictionary characters) is introduced.

Differences in the individual elements of a containeridentification number are also investigated. By using avoting scheme based on a dictionary of owner codes pre-pared in advance to identify the character string in thedictionary to which a character string corresponds [21],owner codes not in the dictionary are not taken as solutionsin recognizing the four English characters representing anowner. In many cases, the final number (check digit) iswritten within the frame line, and the frame line influencescharacter recognition. Thus, character recognition is per-formed by always including a process of eliminating theframe.

Figure 6 shows the entire configuration of the proc-ess.

3.2. Character sequence extraction

An outline of the procedure of character string extrac-tion is as follows.

(1) Find and recognize “U” for a multiplexed binaryimage obtained by using multiple binary thresholds.

(2) Tighten the variability conditions of the directionsand numbers of character strings of a container number byfinding and recognizing “U” as a basis for estimating thebrightness relationship between the characters and back-ground and the character sizes .

(3) If “U” is not found due to deterioration of acontainer, avoid imposing excessive requirements on thequality of the “U” part by performing character stringcandidate extraction [12] on the basis of correlations, whichhas been conventionally used for cut-out vehicle numberareas.

The contents of the process are explained in detailbelow.

3.2.1. Switching presumed conditions ofprocess

It is assumed that the presumed conditions of theprocess can be switched in response to the frequencies ofoccurrence of the kinds of containers at a container gate.This paper presumes the case of a dark character partagainst a background. The process is repeated after switch-ing the presumed condition to the case of a bright characterpart if applying the presumption does not yield a solutionhaving high matchability.

Fig. 6. Image processing and ID mark recognition flow.

42

3.2.2. Image processing for extractingcharacter string

A morphology filter [23] having a kernel with ahorizontal length of (80 × 1) is used to eliminate longcomponents (such as metallic devices on the upper part ofa container) in the horizontal direction and to performbinarization. The result of applying a morphology filterhaving a kernel with a vertical length of (64 × 1) to eliminatelong components (e.g., metallic locking bar) is similarlybinarized. Finally, the theoretical products of these aretaken. Three different binarized thresholds, 15, 30, and 45,are simultaneously generated at 256 gradations, an imageprocessed with a threshold value of 30 is subsequentlyprocessed, and images processed with threshold values of15 and 45 are used if the above image fails.

3.2.3. Character “U” search

After labeling the image of Section 3.2.2, eliminatinglarge labels (in surface area, width, and height), and merg-ing labels, the features (48 dimensions) of weighted direc-tional index histograms [19] are generated and recognitionis performed by the pseudo-Mahalanobis distance method.The condition for finding “U” corresponds to a label witha distance from the character “U” template that is less thansome reference value. When multiple candidates meetingthis condition exist, those located in the upper part havehigher priorities than those in the lower part, those locatedto the right have higher priorities than those located to theleft, and the next candidate is selected if the subsequentprocessing fails.

3.2.4. Character area estimation

From the “U” labels selected in Section 3.2.3, thesizes of the labels taken as subsequent character candidatesare limited. The upper and lower limits of the “U” labelheight are set, and only the upper limit of the width (thelower limit is not set since the width of “1” is narrow) is set.For all labels in which the horizontal central coordinate iswithin a specific range from the horizontal central coordi-nate of the “U” label, vertical writing is inferred if the heightsum exceeds some level, and horizontal writing is inferredotherwise. When horizontal writing is inferred, whether alabel of the size within a standard range exists to the left,right, or low side of the label is checked sequentially andthe character area is determined. In this case, other charac-ters are then processed for character string recognition.When vertical writing is inferred, the character string areais determined by checking the label in the upward–down-ward direction, assuming a string in the vertical direction.

3.3. Character elimination and characterrecognition

The characteristics of the scheme for character rec-ognition by character elimination are as follows.

(1) A dynamic design method that can simultane-ously evaluate optimal character label selection and char-acter recognition is used.

(2) The check digit is used to perform re-recognitionafter mandatory frame operations.

(3) The owner code receiving the most votes in thedictionary voting scheme is selected.

(4) Recognition is performed by using multiplexedbinarized images in order to improve low-quality characterrecognition accuracy. For the final character, the result ofrecognition after frame processing is integrated.

3.3.1. Character recognition processing

Character recognition uses weighted directional in-dex histograms (4 directions, 48 dimensions) as featuresand the recognition scheme uses a pseudo-Mahalanobisdistance method [19]. As English and number data used inconstructing recognition templates, about 10,000 Sin-gaporean vehicle number data photographed previously bythe authors (with maximum variations of about 10 times inthe number of items for each kind of character) and about1000 items of data photographed in a container yard wereused.

3.3.2. Character string recognition bydynamic designing

As the character label coordinate system, the horizon-tal coordinate is assumed to increase from left to right andthe vertical coordinate from top to bottom. In the case ofhorizontal writing, numbering is done from left to rightcorresponding to the bottom left coordinate of the label. Inthe case of multiple steps, labeling is performed succes-sively on the sequential numbers of a character string byadding a certain offset value extending over the steps. In thecase of vertical writing, the numbers are given from top tobottom from the label upper position. The case of horizontallabeling is discussed below. Let the initial label numberconstituting the i-th character be j(i) and the final labelnumber be k(i). k(i) ≥ j(i) and j(i) > k(i – 1). The totalcharacter string recognition evaluation value is

43

Here, l(i, j(i), k(i)) is the character likelihood of the imageobtained by merging the labels from j(i) to k(i) as the i-thcharacter (the value obtained under the assumption that thecharacter likelihood increases as the values obtained bysubtracting a constant value from the calculated distancesfrom English characters for 1 ≤ i ≤ 3, “U” for i = 4, andnumbers for 5 ≤ i ≤ 11 increase). Label merging is done overthe character recognition process by treating a set of multi-ple unconnected labels as one label, and the image thusobtained is called a merged image. The character likelihoodis obtained from a multiplexed binary image by selectingthe minimum value of the distance values of the recognitionresults using binary images and subtracting a constant fromit.

g(i, k(i – 1), j(i)) is the space or gap likelihoodbetween the (i – 1)-th character and the i-th character as anexample. This value is assigned a constant positive valuewithin a certain range and a negative penalty point outsidethat range. Although this procedure is meaningless when i= 1, we let g = 0 in order to combine the addition range withother terms. Furthermore, h(i, k(i – 1) + 1, j(i) – 1) is thecharacter unlikelihood of the image obtained by mergingmultiple labels existing in the space between the (i – 1)-thcharacter and the i-th character into one; it is the evaluationvalue for avoiding skipping a character. Specifically, as anexample, let

and subtract the label character likelihood portion of the gapor space. When k(i – 1) + 1 ≥ j(i), let h = 0. In addition, fori = 1 as with g, let h = 0.

The selection of the m-th optimal character label isrecursively

Here, let L(0, j(0), m(0)) = 0. We solve this recursiverelationship equation by the dynamic design method.

The method of estimating the owner code by voting[representing the i-th character by C(i)] takes the ownercode yielding the maximum value of the voted characterrecognition results as the estimation result. Specifically, ifthe upper n character recognition results of the i-th characterand the corresponding character likelihood evaluation val-ues are represented by R1(i), R2(i), . . . , Rn(i) and V1(i),V2(i), . . . , Vn(i), then

where

The owner code C for which E(C) becomes a maximum isthe recognition result. This amounts to a simplified versionof the “Ambiguous Term Search” [21] used in recognizingsuch items as geographical names.

3.3.3. Frame processing

The frame operation is performed only when amerged image is recognized as the check digit. The contentsof frame processing are explained for the example of amerged image with the left frame line eliminated. Let thevalues after binarizing the character and the frame line be1, the value of the background be 0, and the vertical (Y) andhorizontal (X) directional coordinate ranges of the mergedimage be Y0 ≤ Y ≤ YM and X0 ≤ X ≤ XN. The merged imageis searched from the left edge (X0) to the right, and theposition X that has changed from the character part to thebackground part (from 1 to 0) is obtained. Let the positionX for height direction Y be X(Y), take the histograms of X(Y)for all Y = Y0, Y1, . . . , YM, and let their most frequent valuebe Xs. The left edge of the image is made to be 0 from X0 toXs. Perform a similar procedure on the right, upper, andlower sides of the merged image. Figure 7 shows a concep-tual diagram of the frame process.

3.3.4. Reliability evaluation of recognitionresult

The “unclear” determination is performed in twosteps. The “unclear” determination for one character ismade if l(C2) ÷ l(C1) > r, letting the character likelihood ofthe character kind that usually takes the first place (C1) bel(C1) and the character likelihood of the character kind thatusually takes the second place (C2) be l(C2). r is the“unclear” threshold, for which 0.9 etc. is used. The “un-(4)

(1)

(2)

(3)

44

clear” determination of a character string is made when theevaluation value of Eq. (3) based on dynamic designing isless than a reference value. In the proposed device, in orderto reduce as much as possible both the erroneous recogni-tion and the “unclear” determination of the low-qualitycharacters, the unclear threshold r is defined separately forthe character kinds C1 and C2. Specifically, r, which is notdependent on the kinds of characters, is extended to rC1,C2,which can be defined for combinations of kinds of charac-ters that are determined to be close to the recognition result,and “unclear” is determined if l(C2) ÷ l(C1) > rC1,C2. Thisis because the degree of danger of error of each kind ofcharacter due to the similarity with another character isassumed to differ, and the value of rC1,C is determined fromthe results of identifying learning data so as to minimize thenumber of erroneous recognition × 10 + the number of“unclear” characters [8].

Regarding the check digit, theoretically, it can beconsistently obtained from 10 characters other than thecheck digit. If the 11th character recognition result does notcoincide with the check digit number, determined consis-tently from the other 10 characters, the entire characterstring is clearly not correct, and the character string can bedetermined to be unclear. However, in order to evaluate theperformance of character recognition by eliminating theeffect of knowledge processing, this paper has output non-coinciding cases unaltered, without using the rules regard-ing the check digit.

4. Test Results

4.1. Recognition result

Field tests were conducted using the proposed recog-nition device, installed at a real pier. The tests were con-

ducted from 8:00 am to 5:00 pm for five clear days. Theillumination devices shown in the configuration of therecognition device in Fig. 3 were omitted in the device usedfor the field tests. In addition, filter G was used. Table 2shows the devices used.

Six hundred one data items were obtained in 5 days.Table 3 shows the contents of the combinations of thecharacter and background colors.

Table 4 presents the recognition results. The tableshows the cases in which all container marks of 11 charac-ters were recognized correctly as “correct” cases, and thecases in which even one character is unclear or erroneousas incorrect cases. “One unclear character” indicates casesin which all characters other than the unclear character are

Fig. 7. Box elimination process.

Table 2. Processing devices

Image processing(A/D board)

MHI manufactured board [22]

Number recognitiondevice (PC)

OS: FreeBSD, CPU: PentiumII

333 MHz, memory: 128 Mbyte

Camera Toshiba manufactured CCDcamera IK-542

Camera trigger sensor Hokuyo Denki manufacturedPD1

Table 3. Color combinations of ID mark andbackground

Color ofcharacter

Color ofbackground

Number of items (%)

black white 283 (47.1%)

black gray 116 (19.3%)

black red 2 (0.3%)

yellow brown 11 (1.8%)

white brown 113 (18.8%)

white red 38 (6.3%)

white green 13 (2.2%)

white blue 6 (1.0%)

blue red 6 (1.0%)

dark blue white 8 (1.3%)

black andwhite

green 1 (0.2%)

black brown andwhite

1 (0.2%)

Total 601 (100.0%)

45

correct. Specifically, even a case containing an “unclear”character that contains an error in one place is counted as acase of “erroneous recognition.” An example is a case of “12 3 4 5,” of which “3” cannot be recognized due to deterio-ration and is recognized as noise, and which is recognizedas “1 2 4 5 ?” with a separate mark inserted after “5.” In thiscase, the third and fourth places are erroneously recognized.There were 5 such cases among the 13 cases of “erroneousrecognition.” Twelve cases can be determined to be “un-clear” cases if the rules of the check digit, which were notused in the tests, are used. Furthermore, there were nopartially unclear cases containing more than two unclearcharacters.

Some of the data items included character strings thatwere outside the field of vision, vertically written charac-ters, which have been excluded from this study, and blackcharacter–red background and blue character–red back-ground combinations, which could not be photographedwith good contrast because an R filter was not used. Thedata also contained scratched or contaminated characters

that could not be read by humans, and containers other thanthose studied here. The accuracy was also evaluated foreffective or appropriate data, excluding the data obtainedunder conditions other than those studied here. Table 5shows the results.

Table 6 shows the contents of the data other than theappropriate or effective data and the results of recognitionof this data.

Figure 8 shows an example of a container imagecorrectly recognized by the device presented in this study.

Figure 9 is an example of a container image showingpoor contrast, among the images of inappropriate data.Figure 10 is an example of a tank container image elimi-nated as an inappropriate image. Figure 11 shows the origi-nal image and the processed image incorrectly recognizedamong the appropriate data.

The recognition time is less than about 2 seconds onthe average with the configuration of the devices used in thefield tests (Table 2).

Table 4. Recognition results for whole images

Recognition result Number of items (%)

Correct solution 558 (92.8%)

One character unclear 9 (1.5%)

All characters unclear 21 (3.5%)

Erroneous recognition 13 (2.2%)

Total 601 (100.0%)

Table 5. Recognition results for appropriateimages

Recognition result Number of items (%)

Correct solution 550 (97.9%)

1 character unclear 7 (1.2%)

All characters unclear 1 (0.2%)

Erroneous recognition 4 (0.7%)

Total 562 (100.0%)

Table 6. Inappropriate image descriptions

Contents of data

Recognition result

Total (%)Correctsolution

1 characterunclear

All charactersunclear

Erroneousrecognition

Poor contrast 0 0 7 0 7 (1.2%)

Outside the vision field

Vertical writing 0 0 1 0 1 (0.2%)

Tank container 0 0 1 0 1 (0.2%)

Others that are outside the vision field 1 1 4 1 7 (1.1%)

Outside the vision field total 1 1 6 1 9 (1.5%)

Poor character

Scratching or contamination 6 1 3 7 17 (2.8%)

Hidden due to metallic parts 1 0 2 1 4 (0.7%)

1 character bright–dark reversion 0 0 1 0 1 (0.2%)

Handwritten character 0 0 1 0 1 (0.2%)

Lost character total 7 1 7 8 23 (3.8%)

46

4.2. Discussion

The field test results reveal a recognition accuracy of92.8% for all data and 97.9% for appropriate data. Althoughthis accuracy is sufficiently high as a field accuracy forcontainer mark recognition, further study of unmannedoperation of the system on a gate is needed.

The fact that erroneous recognition reaches severalpercent for inappropriate data is considered an impediment

to automation of the recognition process. In addition, thefield tests may not apply to evaluations in an arbitraryoutdoor environment, since they were conducted on cleardays. The kinds of containers handled by container gatesare unevenly distributed, and tests must be performed undermore varied conditions.

Table 7 summarizes the problems associated withcauses of erroneous recognition and solutions to theseproblems.

Fig. 8. Container image correctly recognized.

Fig. 9. Container image removed as inappropriateimage.

Fig. 10. Tank container removed as inappropriate image.

Fig. 11. Container image incorrectly recognized.

47

5. Conclusions

An automatic container mark recognition device hasbeen developed for implementation in container terminals.The recognition device recognizes a mark on a container byphotographing the back surface of the container in outdoorenvironments with a camera device installed on the con-tainer gate. The camera system has been designed to absorbthe effects of changes in the intensity of illumination due tosunlight and variations in the colors of containers andcharacters. In addition, a recognition scheme for a multi-plexed binarized image with a focus on a dynamic designmethod has been developed in order to handle variations inthe layouts and locations of container mark characterstrings and degradation in characters due to scratches orcontamination.

Evaluation of the performance of the device pre-sented by conducting field tests in a container terminalyielded a recognition rate of 92.8% for all data photo-graphed in a period of 5 days. Data excluding exceptionaldata and scratched or contaminated data that cannot be readby humans is taken as appropriate data. The recognition ratewas 97.9% and the erroneous recognition rate was 0.7% forthe appropriate data.

Although this recognition rate is high for field tests,further study is needed in order to reduce erroneous con-tainer recognition at an unmanned container gate. A major

cause of erroneous recognition is that binarized imagelabels containing images other than characters (e.g.,shadow of a metal rod, part of the frame of a check digit)when converted into character recognition features areclose to items in the dictionary [1]. In recognizing a low-quality character string in outdoor environments, characterrecognition is performed to determine which label is acharacter before identifying the kind of character, and thelabel with the shortest distance to a dictionary item is usedas the character label. In doing so, however, there is a majorproblem of noncharacter noise that influences whether acharacter is “1” or “I.” Thus, even if characters with signifi-cant deterioration are ignored as noncharacters, interpreta-tions that match the required number of places may beallowed. In order to resolve this problem, layout interpre-tation of the position of each character or a determinationof whether the character size or width and the characterluminescence and background luminescence relationshipsmatch those of other characters will be necessary.

In the near future, the authors plan to enhance theapplicability of their container mark recognition device byaddressing the above points and conducting tests includingillumination conditions and filters that were omitted in thisstudy.

REFERENCES

1. Ministry of Transportation. 1999 TransportationWhite Paper—New Developments in Urban Trans-portation Policies for the 21st Century, p 374–384.

2. http://www.samsys.com/art-8-1996.html3. Lee C, Kankanhalli A. Automatic extraction of char-

acters in complex scine images. Int J Pattern Recog-nition Artif Intell 1995;9:67–82.

4. Fos first out of the gate. Cargo Systems, p 446–447,July 1994.

5. Hamada H, Ikeya N, Ito K. Development of automat-ic container mark recognition system. Japanese Me-chanical Engineering Society 73rd Regular MeetingLectures and Papers (IV), p 362–363, 1996.

6. Watanabe I. Containers and international regulations.Kowan, August Issue, p 9–14, 1995.

7. Tsutsumida T, Kido T, Ota K, Kimura F, Iwata A.New developments in the study of character recogni-tion. Current status of handwritten number recogni-tion in postal number data. Tech Rep IEICE1997;PRU96-190.

8. Matsui T, Yamashita I, Wakahara A, Yoshimuro M.The First Character Recognition Technology ContestResults. Symposium Lectures and Papers, p 15–22,1993.

Table 7. Problems and solutions

Problem Causes Solutions

Erroneous recog-nition ofscratched charac-ter

Erroneousrecognition as “1”by binarizing theimage strokes ofmetal parts (inmany cases,vertical rods) inmultiplexedbinary imagerecognition

Character layout(positionalrelations withneighboringcharacters) andbackgroundluminescencecheck

Displacementsdue to a characterunrecognized as acharacter

Determining adeterioratedcharacter as anoncharacter andrecognizing thesubsequentcharacter stringwithdisplacements

Check the layoutof each place ofrecognizedcharacter string

48

9. Fujimoto K, Horino M, Fujikawa Y. Number platereading device based on real time image processing.Omron Technics 1990;30:9–18.

10. Kato H, Nanbu M, Fukui H, Yorita F, Aoki M. Num-ber plate recognition technology. Mitsubishi DenkiGiho 1988;62:9–12.

11. Iida Y, Nakayama H, Miyamoto K, Fujita K, Urata H.Development of high-speed image processing deviceand its applications. Mitsubishi Juko Giho1990;27:76–80.

12. Miyamoto K, Tamagawa M, Fujita K, Hayama Y,Ayaho S. Character string region detection schemebased on correlations. Trans IEICE 1998;J81-D-II:2052–2060.

13. Miyamoto K, Kumano S, Sugimoto K, Tamagawa M,Ayaho S. Recognition scheme for low quality char-acters using multiple features. Trans IEICE1999;J82-D-II:771–779.

14. Miyamoto K, Kumano S, Tamagawa M, Sugimoto K,Urata H. Development of character recognition tech-nology in transportation and shipping industries. Mit-subishi Juko Giho 1996;33:404–407.

15. Kimura F, Tsuruoka S, Shridhar M, Chen Z. Contextdirected handwritten word recognition for postalservice applications. Proc Fifth Advanced Technol-ogy Conference, p 199–213, Washington, DC, 1992.

16. Burges CJC, Ben JI, Denker JS, Lecun Y, Nohl CR.Off line recognition of handwritten postal wordsusing neural networks. Int J Pattern Recognition ArtifIntell 1993;7:689–704.

17. Sakaguchi M. Dynamic designing methods.Shibundo; 1968.

18. Miyamoto K. A study on number plate recognition.Thesis, University of Kyoto, 2000.

19. Harata T, Tsuruoka N, Kimura F, Miyake K. Hand-written Chinese and Japanese character recognitionusing weighted directional index histograms andpseudobase identification method. Tech Rep IEICE1984;PRL83-68.

20. Hagita N, Naito S, Masuda I. Identification of hand-written Chinese characters by contour direction con-tribution feature. Trans IEICE 1983;J66-D:1185–1192.

21. Nakabayashi K, Kitamura M, Kawaoka T. High-speed frame-less handwritten character readingscheme using ambiguous term search. Trans IEICE1991;J74-D-II:1528–1537.

22. Tsutsuminaka T, Mimochi K. Real-time dynamicimage processing device. 4th Image Sensing Sympo-sium Lectures and Papers, 1998.

23. Obata H. Morphology. Corona Press; 1996.

AUTHORS (from left to right)

Shintaro Kumano (member) received his B.S. degree from the Department of Computer Science, University of Tokyo,in 1985 and joined Mitsubishi Heavy Industries, Ltd. His research interests include pattern recognition signal processing. Hereceived an M.S. degree from Georgia Institute of Technology in 1992. He is currently a technical leader at the Takasago Researchand Development Center, Mitsubishi Heavy Industries, Ltd. He is a member of the Japan Society for Nuclear Energy.

Kazumasa Miyamoto (member) received his M.S. degree from the Department of Computer Engineering, Universityof Kyoto, in 1974 and joined Mitsubishi Heavy Industries, Ltd. He was affiliated with the Hiroshima Research Laboratory. Hehas investigated image processing, pattern recognition, and signal processing at the System Technology Development Center(Kobe), Technology Headquarters. He received a D.Eng. degree from the University of Kyoto in 1999. He is currently a leaderat the Takasago Research and Development Center, Mitsubishi Heavy Industries, Ltd. He received a 1991 Kobe City TechnologyContributor Award and 1995 Defense Technology Inventor Award.

49

AUTHORS (continued) (from left to right)

Mitsuaki Tamagawa received his M.S. degree from the Department of Electrical Engineering, Waseda University andjoined Mitsubishi Heavy Industries, Ltd. He has been engaged in research and development on image processing and patternrecognition. He is currently a technical leader at the Technical Laboratory, Takasago Research and Development Center. He isa member of the Image Information Media Society.

Hiroaki Ikeda received his B.S. degree from the Department of Precision Engineering, University of Hiroshima, in 1975and joined Konica (Ltd.). He moved to Mitsubishi Heavy Industries, Ltd. in 1990. His research interests include laser sensorsand other optical applications. He is currently a research leader in the Printing Machines Research Laboratory, HiroshimaResearch Center. He is a member of the Applied Physics Society.

Koji Kan received his M.S. degree from the Department of Electronic Engineering, Osaka Prefectural University, in 1988and joined Mitsubishi Heavy Industries, Ltd. He has worked on the design of electrical devices for ships and other marinemachinery. He is currently a leader at the Electrical Equipment Control Designing and Planning Department, Ships and SeasDivision, Kobe Shipyard and Machinery Works.

50