Automatic Generation of IC Component Configuration Data

HIROTAKE ESAKI, KIYOYUKI KAGII, TAIZO UMEZAKI, and TETSUMI HORIKOSHINagoya Institute of Technology, Japan

SUMMARY

Chip mounters and surface mount device (SMD)inspection systems use image processing techniques for theplacement of SMDs onto printed circuit boards (PCB) andthe inspection of SMDs. Such techniques require the com-ponent configuration data which define the shape of SMDs;however, the creation of this data is currently not auto-mated. The goal of this paper is to offer a system thatgenerates component configuration data automatically byprocessing images of SMDs. There are several target com-ponents, such as IC, BGA (ball grid array), chips, connec-tors, etc., for which data can be generated. In this paper wewill focus on generation of data for IC components. © 2008Wiley Periodicals, Inc. Electr Eng Jpn, 165(4): 76–83,2008; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20686

Key words: configuration data; automatic genera-tion; SMD.

1. Introduction

In recent years, development of electronic deviceshas advanced at a remarkably fast pace. Reduction ofweight and size and expansion of available functionalitiesof devices such as digital cameras, notebook computers,cellular phones, etc., was accompanied by technologicalbreakthroughs in the area of chip mounters (machines forthe placement of electronic components onto printed circuitboards) and of inspection systems for checking the place-ment accuracy. These machines and systems operate usingimage processing principles; they must maintain a highaccuracy in component placement on PCBs and providesufficient capabilities for precise inspection.

The increase in productivity of chip mounters andsystems for surface mount device inspection is an importantissue. An important index of productivity of chip mounters

and systems for the surface mount device inspection is theirthroughput, that is, the number of operations per unit oftime. Gradual upgrading of the processing rate and of therecognition capacity of the image processing techniquesused in these systems also contributes to the increase of thesystem productivity.

Systems for placement and inspection of electroniccomponents on PCBs based on the image processing tech-niques used in these types of equipment require for theiroperation data on configuration of the surface mount de-vices to be prepared beforehand. The placement and inspec-tion of the components are carried out by comparing thecomponent configuration data with the images of actuallymounted components taken by a camera.

Data on the configuration of components can becompiled automatically based on their specifications but atproduction facilities manufacturing electronic products un-der contract it is not so easy to obtain design specifications(due to the proprietary nature of information) and, in manycases, manufacturers have to deal only with actual compo-nents on hand supplied according to orders. In addition, insome cases, a part of the data on the component configura-tion listed in the design specifications is not acquired in theimage, and is not useless for the purposes of image proc-essing systems. Because of this, the data preparation proc-esses remain practically not automated and data necessaryfor the operation of manufacturing systems are compiledmanually in spite of efforts focused on increasing the pro-ductivity of manufacturing systems. Growing sophistica-tion of electronic devices and frequent changes in theproduction process (small-lot production of wide varietiesof products) result in a higher relative cost of preparationof data on configuration of electronic components. In addi-tion, due to the fact that the quality of manually prepareddata can vary depending on individual skills of workersinvolved, they must be trained to employ specific tech-niques to assure uniform quality of prepared data.

This study is undertaken for the purpose of develop-ing a system for an automatic preparation of the componentconfiguration data based on images of actual components

© 2008 Wiley Periodicals, Inc.

Electrical Engineering in Japan, Vol. 165, No. 4, 2008Translated from Denki Gakkai Ronbunshi, Vol. 127-D, No. 2, February 2007, pp. 152–157

76

stored in a computer memory. The goal of the study is toreduce production costs and to upgrade the quality ofproducts by using the computer-based automation of pro-duction processes.

Elements of electronic products to be stored in theproposed database comprise such components as ICs,BGAs (ball grid arrays), small chips, connectors, etc., in-cluding multiple modifications of all components. At theinitial stage, it has been decided to develop a system for theautomatic creation of component configuration data forICs.

2. Component Configuration Data

Data on IC configuration used for the creation ofdatabase in this study are given below. ICs have multipleleads. In order to efficiently express the data on the leads,leads having the same direction, width, length, and pitch aredefined as the lead clusters.

i. P(px, py): coordinates of the lead set [mm]ii. width: lead width [mm]iii. length: lead length [mm]iv. pitch: pitch between the leads [mm]v. num: number of leadsvi. direction: direction of the lead tip [deg]

It is assumed that the reference point (the origin) isin the center of the rectangle circumscribed around thecomponent.

Figure 1 shows an example of an IC with leadsarranged along the four sides of its body. Leads along allsides have the same width, length, and pitch. Therefore,leads arranged in the same direction like those shaded inFig. 1 are grouped together and are defined as one lead set.

Typical configurations of ICs to be investigated inthis study are shown in Fig. 2.

Leads are arranged along either two or four sides ofthe component body and their lengths, widths, and pitchesare usually the same. Moreover, as can be seen in Fig. 2(c),in some cases there are breaks between the lead groupsarranged along the same side.

3. System for Automatic Generation of Data onComponent Configurations

Schematic diagram of the system for automatic gen-eration of data on component configuration is shown in Fig.3.

Electronic components are set in the acquisition ap-paratus connected to a computer where components areexposed to the light and their images are taken by thecamera. Data of the photographed component images aretransmitted to the computer where the images are processedand the component configuration data are automaticallycreated.

During the image processing, IC lead candidates areextracted from the component image data and, after analyz-ing their geometrical arrangement, the configuration dataare created based on the measured dimensions and positionsof the IC leads. Creation of the configuration data is auto-mated by a computer program written according to the

Fig. 1. Configuration data for an IC component.

Fig. 2. Typical configurations of IC components.

Fig. 3. System structure.

77

formalized rules set based on procedures used by humanoperators preparing the configuration data manually.

Flowchart outlining the procedures involved in imageprocessing according to the proposed system is shown inFig. 4.

The proposed method includes the following steps:(1) an electronic component is placed in a horizontal posi-tion in the field of vision of the acquisition apparatuscamera; (2) the rotation correction is performed based onthe information about the edges of the component image.Then, in step (3), a histogram of the image densities ischarted and the digitized thresholds are calculated using theOtsu method [1], after which, using the digital processingand the labeling processing, labels of the lead candidatesare extracted from the image. Next, in step (4), lead candi-dates are clustered and decisions are made regarding thedirection of all leads relative to the reference point of thecomponent (up, down, left, or right). In step (5), accuratemeasurements of positions and sizes of the leads are made.Finally, in step (6), the component configuration data aregenerated.

4. Clustering of Lead Candidates

When data on the IC component configuration aregenerated, it is necessary to group the leads having the samewidth, the same length, and the same pitch in a single set.At this time, it is convenient to know at what sides of thecomponent body the lead candidates are located. Here, wediscuss methods that can be used to discriminate to whichside of the component body a given lead candidate belongs.

4.1 Clustering by the k-means method

In order to determine to which side of the componentbody the lead candidates belong, the extracted lead candi-dates are clustered according to the k-means method [2].

Before proceeding to the clustering, coordinates ofthe lead candidates must be normalized. In order to acquirecoordinates of lead candidates in the image coordinatesystem, it is necessary to check the bounds of the leadcandidates and to normalize the coordinates so that thebounds of both the x- and y-coordinates lie between –1.0and 1.0 (see Fig. 5).

If the minimum and maximum values of the x- andy-coordinates of the coordinate values in the image coordi-nate system p(x, y) are denoted xmin and xmax and ymin andymax, respectively, then the normalized coordinates in thenormalized coordinate system p′(x,y) can be calculatedaccording to the following equations:

The clustering carried out according to the k-meansmethod is intended to arrange the lead candidates into fourclasses corresponding to upper, lower, left, and right sidesof the component (see Fig. 6).

In the process of clustering, distances between eachclass and lead candidate are calculated as Manhattan dis-tances [2] (distances measured along city blocks) betweenpositions of centers of gravity of each class and positionsof centers of gravity of lead candidates. In addition, posi-tions corresponding to the centers of upper, lower, left, andright sides of the component body are considered as theinitial values of the classes in the normalized system ofcoordinates (see Fig. 7).

Clustering experiments were carried out according tothe above method on sample images. For sample images weused images of 24 types of representative IC components.Results of experiments are given in Table 1.

Fig. 4. Outline of processing flow.

(1)

(2)

Fig. 5. Normalization of coordinates of lead candidates.

78

Samples of components having leads in four direc-tions can be clustered in a normal manner. But a normalclustering for samples having leads extended in two direc-tions failed. The reason for the failure is that an attempt ismade to arrange the leads into four classes even if the leadsextend only in two directions.

4.2 Discrimination between the componentshaving two- and four-sided leads

In order to solve the above-mentioned problem, it isdesirable to know in advance whether the analyzed compo-nent has two- or four-sided leads. Then, components withtwo-sided leads are clustered into two classes, and compo-nents with four-sided leads are clustered into four classes.In this study we discriminate based on the variance of thecoordinate values of the lead candidates.

Variances σx, σy of x- and y-coordinates are found bytaking normalized x- and y-coordinates of the system p′(x,y) of the lead candidates in the analyzed image. Variancesof x- and y-coordinates in sample images are shown in Fig.8.

These results suggest that variances σx and σy areconsiderably different depending on whether leads are ar-ranged along either x- or y-axes or along both axes. Ifvariance values are greater than σ = 0.7, it is understoodthat no leads are present in the direction of this coordinateaxis. This condition makes it possible to determine whetherthe target component has leads at two or four sides.

Results of clustering into two classes for two-sidedleads and into four classes for four-sided leads undertakenafter applying this method for the discrimination of com-ponents with two-sided and four-sided leads are shown inTable 2.

Thus, a correct clustering can be achieved for imagesof all sample components by clustering lead candidatesafter discriminating the components into classes of unitshaving two-sided and four-sided leads.

5. Measurement of Lead Positions and Dimensions

5.1 Measurement of the lead position anddimensions by means of the edge detection

In order to create lead set data, positions and dimen-sions of all lead candidates must be accurately measured.Measurements of positions and dimensions are carried outfor tips and bases of all lead candidates by detecting aone-dimensional edge in the width direction. The one-di-mensional edge is detected by observing brightnesschanges at the line defining the edge position. In thismethod, the edge position is considered as the point of anabrupt change of brightness.

Fig. 7. Initial position of classes.

Fig. 6. Classification of the lead candidates.

Table 1. Results of clustering experiments

OK NG

2-sided lead components 0 12

4-sided lead components 12 0

Table 2. Clustering results after discrimination bydirection

OK NG

2-sided lead components 12 0

4-sided lead components 12 0

Fig. 8. Variances of x- and y-coordinates in leadcandidates.

79

Figure 9 shows the concept of detection of the edgeposition of a lead candidate along the A-to-B line. First, thebrightness value is found along the A-to-B line. Then, thebrightness is differentiated and the value by which thebrightness is changed is found. A location corresponding tothe maximum amount of change is considered as the de-tected edge position.

In order to reduce the noise in the actual calculationof brightness, the brightness values are measured as projec-tions on an area of several pixels arranged perpendicularlyto the detection line as shown in Fig. 10, and the brightnessvalues are calculated as average brightness values on thepixels.

Position and size of a lead candidate are measured asfollows: detection points of the one-dimensional (1D) edgeare determined as shown in Fig. 11, then the lead width isdefined as the measurement in the point located at 30% ofthe lead length (which is determined as the distance be-tween the tip and the base of the lead) from the lead tip. Thelocation at 30% of the lead length was selected based onactual values of the lead width observed in devices used inthe experiments.

Using the four measured points, one can find coordi-nates of the lead center in the lengthwise direction andcoordinates of the lead center in the widthwise direction,thus determining the position and the size of the leadcandidate.

By using the positions and sizes of all lead candidatesand directions of all lead candidates found by the k-meansmethod, we grouped lead candidates having the same direc-tions, size, and pitch and created the lead set data mentionedin Section 2.

5.2 Calculation of the lead width detectionpoint

In the previous subsection, the lead width detectionpoint used in the measurements of lead position and sizewas defined as a point located at 30% of the lead lengthfrom the lead tip. If a lead is a perfect rectangle, the leadwidth can be determined at any location, but lead configu-rations are usually more complicated. Since it is not desir-able to detect the lead width at a transitional location, it wasdecided upon examination of configurations of all leadcandidates that this point is the optimal position for deter-mining the lead width.

At first, a width profile of each lead candidate iscalculated by measuring the distance between edges in thedirection of its width while scanning the whole lead in thedirection of its length (as shown in Figs. 12 and 13).Regions of small changes of the lead width are consideredstable and can be used for the lead width detection. There-fore, a portion of the lead width profile characterized bysmall lead width changes was selected for the lead widthdetection.

The above-mentioned operations were applied to alllead candidates and the portions having the smallestamounts of the lead width change were assigned as loca-tions for the lead width detection.

Figure 14 shows the variance of results of the leadwidth detection on sample images for cases when the leadwidth was detected by the above-mentioned method and

Fig. 9. 1D edge detection.

Fig. 11. Detection point of the 1D edge.

Fig. 10. Projection of the 1D edge brightness.

80

when the width was detected at fixed points. The effect ofthis method is not especially noticeable for the samples withan originally low variance, but for the samples with a largeoriginal variance the effect was considerable and the vari-ance was reduced to a certain degree.

6. Experiments on Creation of the ComponentConfiguration Data

Based on the proposed method, we performed experi-ments on the creation of the component configuration datausing the sample images. The experiments were carried outon 24 types of sample images and some results of experi-ments are represented in Figs. 15 and 16. Graphicallyrepresented results are shown as a wire frame of the lead setsuperposed over the original image of the component.

In order to check the adequacy of the created data, weran a program for the component positioning and inspectionby means of the image processing using the actual equip-ment based on the data automatically created by the pro-posed method.

These tests confirmed that the automatically createddata on component configurations performed correctly forall 24 types of components used in the experiments.

7. Conclusions

In this study, we constructed a system for an automat-ic creation of IC component configuration data based on aformalization of manual procedures used for its generation.

The proposed system includes, among other things,a method of increasing the stability of the clustering resultsby discriminating the components into two- and four-sidedunits and by grouping the leads at the time of clustering.Measurements of positions and sizes of leads were carriedout by creating the profile related to the lead width which

Fig. 13. Lead width profile.

Fig. 14. Variances of the lead width detection results.

Fig. 12. 1D edge detection in the direction of leadwidth.

Fig. 15. Results of automatic generation of componentconfiguration (Sample 1).

81

is used to detect a portion of a stable lead width. Weconfirmed that such a portion can serve as the location forthe optimal lead width detection.

Surface mount devices are not limited to ICs only.This class of components includes also ball grid arraydevices, connectors, chips, etc. In this study, we developeda system for automatic generation of the component con-figuration data for ICs. In the future, we plan to design asystem for creation of data on configuration of other typesof components by developing formal rules based on manualprocedures used for the same purpose.

REFERENCES

1. Otsu N. Methods of discrimination and an automaticselection of threshold values based on least squarescriteria. Trans IEICE 1980;J63-D:349–356. (in Japa-nese)

2. Duda RO, Hart PE, Stork DG. Pattern classification.Wiley; 2001.

3. Kurita T. A study on applications of statistical meth-ods to flexible information processing. National In-stitute of Advanced Industrial Science andTechnology.

4. Anderberg MR. Cluster analysis for applications.Academic Press; 1973.

5. Jambu M, Lebeaux M-O. Cluster analysis and dataanalysis. North-Holland; 1983.

6. Kurita T, Otsu N, Abdelmalek N. Maximum likeli-hood thresholding based on population mixture mod-els. Pattern Recognition 1992;25:1231–1240.

7. Shunji M, Talashi H, Hitoshi K. Automatic externalsinspection of LSI wafer multilayer pattern by differ-entiation polarity comparison method. EiC1999;J82-D-II:39–52.

8. Berg M, Schwarzkopf O, Kreveld M, Overmars M.Computational geometry: Algorithms and applica-tions. 2nd ed. Springer-Verlag; 2000.

AUTHORS (from left to right)

Hirotake Esaki (student member) completed the Electronic-Mechanical Engineering course in the Department ofEngineering at Nagoya University in 1995 and completed the first half of his doctoral studies in electronic and mechanicalengineering in 1997. At the present time, he is continuing his doctoral studies at Nagoya Institute of Technology specializingin urban transportation systems. His research interests are in image processing.

Kiyoyuki Kagii (nonmember) graduated in 2004 from the Department of Engineering at Nagoya Institute of Technologymajoring in systems management and engineering. He is now in the doctoral course at the Department of Techno-BusinessAdministration of the Graduate School of Engineering. His research interests are in image processing.

Fig. 16. Results of automatic generation of componentconfiguration (Sample 4).

82

AUTHORS (continued) (from left to right)

Taizo Umezaki (member) graduated in 1982 from Toyohashi University of Technology majoring in information andcomputer science engineering. He received his Ph.D. degree from Nagoya University in 1987 and became an assistant there. In1990, he became an instructor and in 1992 an assistant professor at Chubu University. From 1993 to 1994, he was a visitingresearcher at Carnegie-Mellon University. In 1999, he was appointed a professor at Chubu University. He has been a professorat Nagoya Institute of Technology since 2003. He is engaged in research on speech and image processing, education ofhearing-impaired children, human-care robots, etc. He holds a D.Sc. degree (engineering), and is a member of the InformationProcessing Society of Japan, the Acoustical Society of Japan, IEICE, Welfare Engineering Society, and Human Interface Society.

Tetsumi Horikoshi (nonmember) graduated from Hokkaido University in 1973. After completing his graduate studies atTokyo Institute of Technology, he joined Japanese National Railways. Later, he worked as an assistant at Toyohashi Universityof Technology and as an instructor at Osaka City University. Currently, he is a professor in the Department of Techno-BusinessAdministration, Graduate School of Engineering, Nagoya Institute of Technology. He is involved in research works in suchareas as environmental psychology, occupational ergonomics, and environmental design. He holds a D.Sc. degree (engineering),and is a member of the Society of History of Industry and Technology of Japan, Architectural Institute of Japan, JapaneseInstitute of Environmental Management, and Society of Heating, Air-Conditioning and Sanitary Engineering of Japan.

83