Spectral vision system for measuring color images

2352 J. Opt. Soc. Am. A/Vol. 16, No. 10 /October 1999 Hauta-Kasari et al.

Spectral vision system for measuring color images

Markku Hauta-Kasari

Department of Information Technology, Lappeenranta University of Technology, P.O. Box 20,FIN-53851 Lappeenranta, Finland

Kanae Miyazawa and Satoru Toyooka

Graduate School of Science and Engineering, Saitama University, 255 Shimo-okubo, Urawa,Saitama 338-8570, Japan

Jussi Parkkinen

Department of Computer Science, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland

Received October 16, 1998; revised manuscript received March 19, 1999; accepted June 10, 1999

We present a spectral vision system that can be used to measure a color spectrum and two-dimensional spec-tral images. First, a low-dimensional color filter set was designed by an unsupervised neural network. Thena compact optical setup for the spectral synthesizer was constructed to synthesize the light that corresponds tothe spectral characteristics of the color filter. In the optical setup a liquid-crystal spatial light modulator wasused to implement color filters. A sample was illuminated by the synthesized lights, and the intensity imagesthat correspond to the inner products between the color filter and the sample were detected by a CCD camera.From the detected inner products the sample’s color spectra were reconstructed by use of a pseudoinverse ma-trix. Experimental results of measuring a single color spectrum and spectral images are presented. © 1999Optical Society of America [S0740-3232(99)01910-9]

OCIS codes: 330.0330, 330.1710, 330.6180, 330.1720, 300.6170.

1. INTRODUCTIONMultispectral imaging has received a great deal of atten-tion recently. Spectral measurements are used, for ex-ample, in remote sensing,1 computer vision, and indus-trial applications.2 Spectral information has become animportant quality factor in many industrial processes be-cause of its high accuracy.

In color research the color analysis is usually done withthree-dimensional color coordinate systems such as CIExyY, CIELAB, CIELUV, and red–green-blue (RGB) colorspaces. In the human color vision system there are threetypes of photoreceptors3,4 and for this reason these three-dimensional color coordinate systems are usually used incolor representation. These models are computationallyeffective and suffice for many purposes, but they haveproblems such as metamerism,5 where the same three-dimensional color coordinate corresponds to several dif-ferent spectra. If the measured color spectrum coveringthe visible spectral region from 380 to 780 nm is used asthe color representation, then metamerism is avoided andaccuracy is high. When the spectral imaging system istuned to measure the visible light, then the measured im-age represents a high-quality color image, in which everypixel contains a color spectrum.

To measure a color spectrum, a device such as a mono-chromator, a radiometer, or a spectrophotometer is usu-ally used. Spectral images can be measured, for ex-ample, by a CCD camera with narrow-band interferencefilters,6 by an acousto-optical tunable filter,7 or byFourier-transform-based methods.8 These devices are

0740-3232/99/102352-11$15.00 ©

usually expensive, and a large amount of image datamust be processed and stored. The spectra are generallymeasured from 1- to 10-nm intervals, and therefore, forexample, the spectral image measured in the wavelengthrange from 400 to 700 nm contains from 301 to 31 compo-nent images, respectively. Transmission of the spectralimage obtained by these conventional methods is difficultbecause of the large amount of data to be transmitted.

It has been shown that low-dimensional representationof spectra can reproduce the original spectrum accurately.One way to compress spectra is to obtain the measuredspectra and then compress them by software.9,10 An-other approach is to design the low-dimensional multi-spectral imaging system such that one already has ac-quired optimal component images for spectralreconstruction.

Recently, low-dimensional multispectral imaging sys-tems have been the focus of growing interest.Tominaga11 proposed a multichannel vision system basedon the use of a CCD camera and six color filters. The sys-tem was used to reconstruct the surface spectral reflec-tance and illuminant spectral power distribution from theimage data. Baronti et al.12 used a multispectral imag-ing system with 29 filters to analyze works of art in awavelength range from 420 to 1550 nm. Haneishi et al.13

designed five color filters for archiving spectral images ofartworks. Lenz et al.14 designed low-dimensional colorfilter sets for color spectra by optimizing an energy func-tion based on second- and fourth-order statistical mo-ments.

1999 Optical Society of America

Hauta-Kasari et al. Vol. 16, No. 10 /October 1999 /J. Opt. Soc. Am. A 2353

Spectra of natural color samples are smooth and thespectral components correlate highly with each other.Parkkinen et al.9 showed that a spectral database con-taining 1257 samples measured from the Munsell book ofcolor15 can be represented accurately by a few basis vec-tors produced by the subspace method. These basis vec-tors can also be used to describe natural color spectra.16

Jaaskelainen et al.17 implemented the learning subspacemethod optically. They used a liquid-crystal (LC) spatiallight modulator to implement the color filters correspond-ing to the basis vectors. Hayasaka et al.18 developed thesystem described in Ref. 17 to analyze two-dimensionalmicroscopic images.

In a previous study Hauta-Kasari et al.19 designed alow-dimensional color filter set with a constraint of posi-tive spectral values by an unsupervised neural networkfor 1269 color spectra measured from the Munsell book ofcolor. In this paper we propose a low-dimensional spec-tral vision system that can be used to measure a colorspectrum and two-dimensional spectral color images. Inthe proposed spectral vision system the sample is illumi-nated by synthesized lights that correspond to the de-signed color filters. The color filters are implemented op-tically by the use of a LC spatial light modulator, and theintensity images of the filtering process are detected by amonochrome CCD camera. The detected intensity im-ages correspond to the optically calculated inner productsbetween the color filter set and a sample. From the op-tically calculated inner products the sample’s spectra isreconstructed by use of a pseudoinverse matrix. In theexperimental part of this paper the results of acquiringthe transmittance spectra of color sheets and a two-dimensional spectral image of a real-world sample arepresented. The proposed method is fast, and the amountof data obtained from the filtering process is small andtherefore convenient for storing and transmitting thespectral image.

The paper is organized as follows. In Section 2 webriefly review the color filter design from our previousstudy.19 Then in Section 3 we introduce the optical setupfor the spectral synthesizer and the experimental setupfor the spectral vision system. In Section 4 we show theexperimental results of our measurements, and in Section5 we discuss our results.

2. COLOR FILTER DESIGNThe subspace method,20 which is based on the Karhunen–Loeve expansion, can be used to define a low-dimensionalbasis to describe the spectral data accurately. The colorspectra measured from the Munsell book of color can berepresented accurately by 3–8 basis vectors produced bythe subspace method.9 The basis vector set for the Mun-sell color spectra is orthogonal and contains negative co-efficients, which cannot be directly implemented optically.In Refs. 17 and 18 the basis vector set was biased and wasmultiplied to make it suitable for optical implementation.In our previous study,19 using an unsupervised neuralnetwork, we designed a low-dimensional color filter setcontaining only positive coefficients for the 1269 Munsellspectra. The competitive learning algorithm was basedon the Instar algorithm of Grossberg,21 which was incor-

porated by Kohonen’s22 self-organizing map with thewinner-take-all layer. The neural network clusters thecolor spectra, and after learning, the centers of the clus-ters are used as color filters. A detailed description ofcompetitive learning and self-organization can be foundin Refs. 21–23.

Figure 1 shows the color filter set of four broadbandcolor filters that are designed by the unsupervised neuralnetwork. In the experimental part of this paper, color fil-ter sets of four, five, and six filters are used. Note that bythe use of an unsupervised neural network we can designdifferent numbers of color filters depending on the accu-racy needed in each application. In Ref. 19 we showedthat the Munsell spectral database was reconstructed bythe designed color filters with sufficient accuracy, and thereconstruction accuracy was comparable with that of theKarhunen–Loeve-transform-based subspace method.

The designed color filter set is nonorthogonal, and inreconstructing a spectrum s with it, a pseudoinverse ma-trix can be used:

s8 5 W~WTW !21WTs, (1)

where W is the filter set. In the optical implementation,W(WTW)21 is known, and the inner products WTs be-tween the filter set W and the sample’s spectrum s are de-termined experimentally. The filter effect on the samplecan be produced either by filtering a reflecting or a trans-mitting light of the sample or by illuminating the sampleby synthesized light with the spectral characteristics ofthe filter.

3. OPTICAL SETUPThe inner products WTs in Eq. (1) between a broadbandcolor filter set W and a sample s can be calculated opti-cally by the use of a liquid-crystal (LC) panel.17,18 If thesample is illuminated by synthesized light that has thespectral characteristics of the color filter Wi , then the de-tected intensity of the sample corresponds to the innerproduct Wi

Ts. The optical setup of the calibration mea-surements for the spectral vision system is shown in Fig.2. We use this setup to measure the characteristics ofthe system and to set parameters for the final spectral vi-sion system. The optical setup is discussed in detail inthe following sections.

Fig. 1. Filter set of four learned filters used in the proposedspectral vision system.


A. Spectral SynthesizerTo synthesize the light corresponding to the color filter,we constructed the optical setup for the spectral synthe-sizer shown in Fig. 3. The white-light source is a halogenlamp pair with two 150-W lamps. The light is introducedto a slit by a fiber light guide, which is omitted in Fig. 3,and then is reflected by a mirror and is incident on a con-cave grating. The collimated light is dispersed on the fo-cal plane of the concave grating. On the dispersion planethere are a rectangular window, a cylindrical lens, and aliquid-crystal panel, LC. The transmittance of the LCpanel along the wavelength axis is controlled by a com-puter through a monochrome image board and a LCdriver. The light passing through the LC panel is finallymixed by the second concave grating. The function of thecylindrical lens in the dispersion plane is to gather lightenergy effectively to the second grating, in order to pro-vide good mixing and to make the light loss as small aspossible. Mixed light from the second grating is directedto the measuring plane by a mirror. The real size of theoptical setup for the spectral synthesizer is 30 cm3 15 cm, with a height of 7 cm.

B. Characteristics of the Liquid-Crystal PanelThe designed color filters are implemented optically bythe use of a LC panel. To control the transmittance ofthe LC panel along the wavelength axis accurately, onemust know the spectral characteristics of the LC panel.

The LC panel is the key device in our spectral vision sys-tem, and therefore we carefully investigated its character-istics.

The transmittance of the LC panel, which is a compo-nent of the commercial Sharp XV-NV1 projector, is con-trolled by computer through a monochrome image boardhaving 512 3 640 pixels. The digital signal correspond-ing to the transmittance pattern containing the input lev-els between 0 and 255 is transferred from the computer’simage board to the LC driver, which sends the video out-put to the LC panel. The screen size of the LC panel is19.8 mm 3 26.4 mm, containing 624 3 832 pixels. Fig-ure 4 shows a schematic drawing of how the LC panel iscontrolled. The LC panel is an active matrix type withthin-film transistors. This type of LC panel is known tobe free of cross-talk phenomena, which means that thetransmittance of a single pixel is not affected by thetransmittance of the surrounding pixels. We also con-firmed this experimentally.

First, we measured the light-source spectrum of thehalogen lamp that we use in our optical setup shown inFigs. 2 and 3. The transmittance of the LC panel was setto an input level of 255, and the output spectrum wasmeasured by a CCD camera (SONY XC-73) through 31narrow-band interference filters covering the spectralrange from 400 to 700 nm at 10-nm intervals. The trans-mittance data for the 31 narrow-band interference filterswere measured in advance by a Shimadzu UV-VIS

Fig. 2. Optical setup of the calibration measurements for the spectral vision system.

Fig. 3. Optical setup of the spectral synthesizer.


2500PC spectrophotometer. The raw data of the mea-sured light-source spectrum included a serration causedby the nonuniform transmittance of the narrow-band fil-ters, and therefore the raw data were divided by the areaunder each transmittance curve of the interference filters.Figure 5 shows the spectrum of the light source. It canbe seen that the spectrum is smooth, but it has low inten-sities in the wavelength areas from 400 to 410 nm andfrom 650 to 700 nm. The reason for this is the insuffi-cient performance of the halogen lamp in the beginning ofthe blue area of the spectrum. The halogen lamp housethat we used contains heat absorbance filters to cut offthe near-infrared and infrared areas of the radiation to

Fig. 4. Schematic drawing of control of the LC panel.

Fig. 5. Measured light-source spectrum.

Fig. 6. Spectral characteristics of the LC panel.

keep the increasing temperature of the illuminant at asuitable level, and therefore the wavelength area of 650 to700 nm has a low intensity of light.

Next we investigated the spectral characteristics of theLC panel. In the following experiments the output lightfrom the spectral synthesizer was introduced to the mono-chromator by a fiber light guide, was detected by a photo-multiplier tube, and was finally stored on the computerthrough a 12-bit analog-to-digital converter.

We analyzed the spectral characteristics of the LCpanel as follows. We changed the whole LC-panel trans-mittance from the input level of 0 to 255, at one-input-level steps. The 256 output spectra were measured bythe monochromator. Figure 6 shows a part of the resultsfor the wavelength bands 420, 520, and 620 nm, wherethe data is normalized to the maximum transmittancevalue of each curve. The curves for 420 and 620 nm con-tain some fluctuation, which is caused by the low signal-to-noise ratio in these wavelength areas. The reason forthe low signal-to-noise ratio is the insufficient illumina-tion conditions in these wavelength areas, as can be seenfrom Fig. 5. The spectral characteristics of the LC panelshown in Fig. 6 indicate that the transmittance of the LCpanel is almost constant for every curve below the inputlevel of 100. Between the input levels of 100 and 255 thecurves are almost linear. We can also see from Fig. 6that in the linear region above the input level of 100, thetwo curves of 420 and 620 nm are shifted by 20 input lev-els. From these experiments we concluded that color fil-ters can be implemented on the LC panel between the in-put levels 100 and 255, including the linear shift of 20input levels.

4. EXPERIMENTSA. Controlling the Liquid-Crystal PanelIn our experiments we use the wavelength range from400 to 700 nm. In the dispersion plane the width of thelight between 400 and 700 nm is 12.4 mm. To implementthe color filter in this area, one must know exactly thecorrespondence between the location of LC panel and thewavelength of the light. In the following procedure thepixel numbers correspond to the pixels in the mono-chrome image board. Note that the input levels arechanged only along the wavelength axis (from 1 to 512pixels), while along the height axis (from 1 to 640 pixels)the input levels are kept constant. Therefore we showthe transmittance patterns one dimensionally in the fol-lowing sections.

We produced the transmittance pattern on the LCpanel where only a narrow slit of 20 pixels near the bluearea was transparent. The output spectrum was mea-sured by the monochromator. A similar pattern was pro-grammed to the red area, and the output was measuredby the monochromator. From these results we calculatedthe location for the 400- and 700-nm light by linear linefitting. The calculated location for the 400-nm wave-length was pixel number 111 and for the wavelength of700 nm was pixel number 415.

B. Synthesizing the LightIn the next experiment we checked the accuracy of syn-thesizing the light. We used a color filter set of four fil-


Fig. 7. (a) LC-panel transmittance patterns for the set of four filters. (b) Optically measured results of the filtered illuminator. Solidcurves, designed filters multiplied by the light-source spectrum; dotted curves, measured results.

Fig. 8. Spectra of six transparent samples measured by the spectrophotometer (solid curves) and the spectra measured by the spectralvision system with four filters (dotted curves).

ters, shown in Fig. 1. The choice of the optimal dimen-sion for the filter set in our spectral vision system isdiscussed in more detail in Subsection 4.C.

We prepared the transmittance patterns for the colorfilter set as follows. The filter set was first interpolatedbetween pixels 111 and 415 and then scaled between in-put levels 100 and 235. The linear shift from input level1 to input level 20 along the spectral axis was applied tothe filter set. The final filter set is therefore between the

input levels 100 and 255. See Subsection 3.B for a de-tailed discussion of the LC-panel characteristics. Figure7(a) shows the LC-panel transmittance patterns for theset of four filters. These patterns were programmed oneby one to the LC panel, and the output spectra were mea-sured by the CCD camera with 31 narrow-band filters.For each filter the output intensity was detected as an av-erage intensity inside the 10 3 10 pixel image window.Figure 7(b) shows the results of the measurements, where


the solid curves are the designed filter set multiplied bythe light-source spectrum shown in Fig. 5 and the dottedcurves are the measured output spectra when this lightsource is used. It can be seen that the system can syn-thesize the illumination corresponding to each color filterwith sufficient accuracy.

The error in filter 4 is caused by the low intensity of thesynthesized light. As shown in Fig. 1, filter 4 has low in-tensity in a nearly flat region in the wavelength area from400 to 600 nm. In our experiments we noticed that thelights with low intensities were difficult to implement bythe use of the LC panel. See also the spectral character-istics of the LC panel in Fig. 6, where the transmittance

Table 1. Averaged CIE xy and CIE L* a* b* Errorsover Six Transparent Color Samples for the

Spectra Measured by the Spectrophotometerand by the Spectral Vision System

Number ofFilters

AveragedCIE xy Errors

(Dx, Dy)

AveragedCIE L* a* b* Errors

(DE* )

3 0.0691 25.304 0.0324 10.135 0.0396 10.686 0.0350 10.90

Fig. 9. Spectral vision system for acquiring spectral images.

Table 2. Comparison of the CIE xy and CIE L*a*b* Errors over Six Transparent Color Samples forSpectra Measured by the Spectrophotometer (s) and Spectra Measured by the Spectral Vision System (s8)

with Four Filters

SampleNumber

CIE xy CIE L* a* b*

s s8Error

(Dx, Dy) s s8Error(DE* )

1 0.2558 0.2336 0.0222 36.61 32.93 8.870.3938 0.4226 0.0289 227.01 234.85

5.88 7.802 0.2571 0.2172 0.0399 38.23 35.61 16.68

0.4974 0.5716 0.0742 242.95 258.5621.66 26.91

3 0.4117 0.4380 0.0263 40.28 37.59 9.590.5445 0.5199 0.0247 217.79 29.07

56.31 53.384 0.5268 0.5458 0.0190 30.87 28.54 4.00

0.4123 0.4024 0.0099 20.96 24.1739.28 38.77

5 0.3239 0.2952 0.0286 40.17 39.21 8.400.6116 0.6427 0.0311 242.87 251.20

52.36 52.736 0.1988 0.1410 0.0578 31.76 27.10 13.23

0.3120 0.2853 0.0266 225.68 236.40210.67 216.88

Average 0.0324 10.13


curves begin to increase after the intensity level of 100.Filter 4 has intensities near the value 100 with lowsignal-to-noise ratios. We tried to correct this by makingfilter 4 brighter, but when the intensity scale becomesnarrower (for example from 150 to 255), it limits the ac-curacy of other synthesized lights.

C. Dimension EstimationNext we investigated experimentally the optimal dimen-sion for our model, i.e., how many filters should be used.In the following experiment we used transparent colorsamples, which were prepared by taking positive slides ofMunsell color chips.17 A transparent color sample wasplaced in front of the second mirror (see Fig. 3) and wasilluminated by the synthesized lights, and the intensitydata that correspond to the inner product between thecolor filters and sample’s color spectrum were detected bythe CCD camera. The intensities were detected as aver-age intensities inside the 10 3 10 pixel image window.The sample’s spectrum was then reconstructed by usingthe pseudoinverse matrix in Eq. (1). The spectra of thesamples were also measured in advance by a ShimadzuUV-VIS 2500PC spectrophotometer so that the resultscould be compared. In these experiments the spectrawere sampled at 2-nm intervals from 400 to 700 nm.

In theory, also confirmed by computer simulations, ifthe dimension of the filter set W in Eq. (1) increases, thenthe estimation error between the sample’s true spectrums and the reconstructed spectrum s8 decreases. How-ever, in the series of experiments with transparent colorsamples, we noticed that when more than four filters wereused, the estimation error began to increase rapidly. Weinvestigated the reason for this and found that the in-verse matrix in Eq. (1) was sometimes near singular, andthen a very small error between the optically calculatedinner product and the theoretical inner product causedlarge reconstruction errors. In those cases we used aregularization technique based on the truncated singular-value decomposition to avoid the effect of near singularityin the spectrum reconstruction.

Figure 8 shows the spectra of six transparent colorsamples that were acquired with the set of four filters.We calculated the CIE xy and CIE L* a* b* errors for thesix transparent color samples used in our experiments.We cut the negative coefficients of the reconstructed spec-tra to zero, and in the color coordinate calculations theCIE 1931 x, y, and z color-matching curves and the stan-dard light source D65 were used. The averaged CIE xyand CIE L* a* b* errors for the sets of three, four, five,and six filters are tabulated in Table 1, where the DE*value is defined as DE* 5 (DL* 2 1 Da* 2 1 Db* 2)1/2.The inverse matrix in Eq. (1) was near singular in thecase of the sets of five and six filters, and in those casesthe truncated SVD was used. By using the truncatedSVD we managed to lower the reconstruction errors DE*from a value of 20 to value near 10. Table 1 shows thatthe error DE* between the sets of four, five, and six filtersis smaller than DE* 5 1, and therefore we decided to usethe filter set with the lowest dimension, i.e., the set of fourfilters. Table 2 shows the CIE xy and CIE L* a* b* errorsfor the six transparent color samples shown in Fig. 8.

D. Spectral ImageFinally, we acquired a spectral image from a real-worldobject, using the spectral vision system shown in Fig. 9.A setup of a strawberry and a mandarin lying on a tablein front of a colored panel was used as the real-worldscene. In this experiment we measured the sample’s re-flectance spectra. The size of the synthesized light areain the measuring plane was 8 cm 3 5 cm.

The sample used is shown in Fig. 10(a) as a real-sizegray-level image of 397 3 290 pixels. The background ofthe sample are color sheets, which are, from left to right,painted blue, painted green, glossy yellow, and glossy red.We illuminated the sample by four synthesized lights cor-responding to the color filters shown in Fig. 1 and de-tected the reflected-intensity images with the CCD cam-era. The detected-intensity images are shown in Fig.10(b).

From the detected intensities we reconstructed thespectral image in the wavelength range from 400 to 700nm at 10-nm intervals, using a pseudoinverse matrix.Figure 11(a) shows the wavelength bands from 430 to 650nm at 20-nm intervals of the spectral image acquired bythe spectral vision system with four filters. To comparethe results, we measured the same spectral image withthe CCD camera, using 31 narrow-band interference fil-ters covering the wavelength range from 400 to 700 nm at10-nm intervals. Figure 11(b) shows the measured re-sults at the wavelength range from 430 to 650 nm at20-nm intervals.

Figure 12 shows examples of spectra at different loca-tions of the spectral image. The spectra in the top roware, from left to right, painted blue color sheet, paintedgreen color sheet, and glossy yellow color sheet. Thespectra in the bottom row are, from left to right, glossyred color sheet, strawberry, and mandarin. To give animpression of the colors in the spectral images, we con-verted them to RGB color images. The negative coeffi-cients of the reconstructed spectra were cut to zero, andthe RGB color coordinates were calculated by using CIE1931 x, y, and z color-matching curves and the standardlight source D65. Figure 13(a) shows the RGB image forthe spectral image measured by our spectral vision sys-tem with four filters. The RGB image for the spectral im-age measured by the CCD camera with 31 interference fil-ters is shown in Figure 13(b).

5. DISCUSSIONWe have presented a spectral vision system that can beused to measure a color spectrum and two-dimensionalspectral images. First we designed a low-dimensionalcolor filter set, using an unsupervised neural network.Then we constructed a compact optical setup for the spec-tral synthesizer. The spectral synthesizer can also beused in other applications where a certain illuminationmust be synthesized. We illuminated the sample withsynthesized light, and therefore our present system islimited to indoor measurements. In outdoor use a morepowerful illuminator is needed, and calibration must beperformed frequently because of the change in illumina-tion over time.


We implemented the color filters optically by the use ofa LC panel. The thin-film-transistor type of LC panel isfree of cross-talk phenomena and therefore is easy to con-trol. We used a halogen lamp pair as a light source. Ithas low light energy in blue and red areas, but the shapeof its spectrum is smooth and suitable for filtering on thedispersion plane. Another possible light source is, for ex-ample, a xenon lamp, which has more light energy in blueand red areas but also some sharp peaks in the visiblelight area, which can cause problems with filtering on thedispersion plane. In the present system the energy of thesynthesized output light was suitable for detection by theCCD camera when the sample was located at a distance of40 cm from the spectral synthesizer. The area of light inthe measuring plane was 8 cm 3 5 cm. To make the dis-tance between the spectral synthesizer and the samplelonger and the light area larger, one should use a lightsource of higher light energy or a high-sensitivity camera.

We determined experimentally the optimal dimensionfor the color filter set to be used in the spectral vision sys-tem. In theory, if the dimension of the filter set W in Eq.(1) increases, then the estimation error between the sam-ple’s true spectrum s and the reconstructed spectrum s8decreases. In the experiments, however, we noticed that

Fig. 10. (a) Sample as a real-size gray-level image, illuminatedby a halogen lamp. (b) Detected intensity images of the sample,illuminated by synthesized lights, that correspond to the fourcolor filters.

when more than four filters were used, the estimation er-ror began to increase rapidly. Tominaga11 also showedthat the appropriate linear model dimension depends onthe properties of the illuminants and on the properties ofthe measurement instrument. Furthermore, the dimen-sion of the linear model may be limited, for example, bythe possible noise in the optical system and by the spec-tral sensitivity of the CCD camera. In our experimentswe measured the spectra of the transparent color sheetsusing different number of filters, and we concluded thatfour was the optimal number of filters in our present sys-tem. We also noticed that when the inverse matrix inEq. (1) was near singular, then a very small error be-tween the optically calculated inner product and the the-oretical inner product caused large errors in the spectrumreconstruction. In order to avoid the problem of near sin-gularity, we used a truncated singular-value-decomposition method. The filter set designed by the un-

Fig. 11. (a) Spectral image measured by the spectral vision sys-tem with four filters. (b) Spectral image measured by the CCDcamera with 31 narrow-band interference filters.


Fig. 12. Spectra at six locations of the spectral image. Solid curves, spectral image measured by the CCD camera with 31 narrow-bandfilters; dotted curves, spectral image measured by the spectral vision system with four filters.

Fig. 13. Spectral images converted to RGB images. (a) Spectral image measured by the spectral vision system with four filters; (b)spectral image measured by the CCD camera with 31 narrow-band filters.

supervised neural network contains uncorrelated color fil-ters, but inside the light-source spectrum of the halogenlamp the independence becomes weaker.

In our final experiment we acquired a spectral imagewith our spectral vision system, using four filters. Thisimage compared with the spectral image measured by theCCD camera with 31 narrow-band interference filters.The spectra measured by the two methods correlatedwell. It can be seen from the Fig. 12 that the smoothspectrum of the mandarin is approximated by our methodas the spectrum containing two peaks. The same effectcan be seen from the Fig. 11, where the wavelength bandsof 550 and 570 nm obtained by our method are darker

than those obtained by the narrow-band filters. Thesame effect can be seen in transparent color sample 3 ofFig. 8. We believe that the reason for this comes fromthe problem of near singularity discussed in the previousparagraph.

After measurement of the spectral image with ourspectral vision system, the quality of the imagescan be improved by several methods reported in theliterature. These methods can be used, for example, toinvestigate a dichromatic reflection model and the specu-lar component in the images,11 the spectral sensitivity ofthe CCD camera,24 and the highlights in images.25

These methods are not included in this study.


Many of the present spectral imaging systems arebased on line scanning.2 In these systems the spectraare measured with arbitrary accuracy. The complexity ofthe acquisition time for each line is O(1) and for thewhole image O(N), where N is the number of lines in the-image. Extra care should be taken in the acquisition ofequilateral pixels when either the object or the camera ismoving. In the proposed spectral vision system, the spa-tial resolution of the image is defined by the CCD array,and one can obtain a two-dimensional image directly.The different spectral components are, however, acquiredseparately. In this case the time complexity for acquir-ing a component image is O(1), and for the whole spec-tral image the time complexity is O(L), where L is thenumber of spectral components and L ! N. Our belief isthat the proposed system is easier to use and is a betterchoice for static objects. For moving objects the numberof spectral components and the speed of image acquisitionimpose limits of use. The most critical part is the setuptime of the LC spatial light modulator.

We have shown that our spectral vision system can beused to measure spectral images. The data obtainedfrom the filtering process consists of only four mono-chrome images, which can be used to reconstruct thespectral image by a pseudoinverse matrix. The acquireddata are convenient for storing and transmitting the spec-tral image. The optical system is used to calculate theoptical inner product, and therefore this system can beused in various optical pattern recognition tasks, for ex-ample in classifiers, where the classification criteria con-tain the inner product calculation. There are still someopen questions regarding our system, for example, thechoice of color filter set, possible system noise, light en-ergy, and the size of the sample. The main result of thispaper is a prototype of the spectral vision system, whichcan be developed further for more accuracy in its colorrepresentation.

ACKNOWLEDGMENTSThis work was done when Markku Hauta-Kasari was vis-iting researcher at the Graduate School of Science andEngineering, Saitama University, Japan. MarkkuHauta-Kasari was supported by scholarships from Mon-busho (Japanese Ministry of Education) and fromLappeenranta University of Technology, Finland. Thiswork was also partly supported by a Grant-in-Aid for De-velopment of Scientific Research (B) (10555013) of theJapanese Ministry of Education, by the Wihuri Founda-tion, Finland, and by the Emil Aaltonen Foundation, Fin-land. The transmittance data for the interference filtersand the spectra for the transparent samples were mea-sured by a Shimadzu UV-VIS 2500PC spectrophotometerat the Institute of Physical and Chemical Research(RIKEN), Wako, Japan. The Munsell spectral databaseis available at the www server of the Lappeenranta Uni-versity of Technology, Finland, http://www.it.lut.fi/research/color/lutcs–database.html. We appreciate thefruitful discussions of this paper with Shigeki Nakauchi.

Address correspondence to Markku Hauta-Kasari,Department of Computer Science, University of Joensuu,

P.O. Box 111, FIN-80101, Joensuu, Finland. Telephone,358-13-251111; fax, 358-13-2513290; e-mail,[email protected].

REFERENCES1. S. M. Ramasamy, V. Venkatasubrmanian, and S.

Anbazhagan, ‘‘Reflectance spectra of minerals and theirdiscrimination using Thematic Mapper, IRS and SPOTmultispectral data,’’ Int. J. Remote Sens. 14, 2935–2970(1993).

2. T. Hyvarinen, E. Herrala, and A. Dall’Ava, ‘‘Direct sightimaging spectrograph: a unique add-on componentbrings spectral imaging to industrial applications,’’ inProceedings of IS&T/SPIE Symposium on Electronic Imag-ing (SPIE, Bellingham, Wash., 1998), Vol. 3302-21,pp. 165–175.

3. P. K. Kaiser and R. M. Boynton, Human Color Vision,2nd ed. (Optical Society of America, Washington, D.C.,1996).

4. G. Wyszecki and W. S. Stiles, Color Science: Concepts andMethods, Quantitative Data and Formulae (Wiley, NewYork, 1982).

5. M. S. Drew and B. V. Funt, ‘‘Natural metamers,’’ CVGIP:Image Understand. 56, 139–151 (1992).

6. S. Kawata, K. Sasaki, and S. Minami, ‘‘Component analysisof spatial and spectral patterns in multispectral images. I.Basis,’’ J. Opt. Soc. Am. A 4, 2101–2106 (1987).

7. J. Hallikainen, J. P. S. Parkkinen, and T. Jaaskelainen,‘‘Color image processing with AOTF,’’ in Proceedings of the6th Scandinavian Conference on Image Analysis, M. Pieti-kainen and J. Roning, eds. (Pattern Recognition Society ofFinland, Oulu, Finland, 1989), pp. 294–300.

8. K. Itoh, ‘‘Interferometric multispectral imaging,’’ inProgress in Optics XXXV, E. Wolf, ed. (Elsevier, Amster-dam, 1996), pp. 145–196.

9. J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen,‘‘Characteristic spectra of Munsell colors,’’ J. Opt. Soc. Am.A 6, 318–322 (1989).

10. S. Usui, S. Nakauchi, and M. Nakano, ‘‘Reconstruction ofMunsell color space by a five-layer neural network,’’ J. Opt.Soc. Am. A 9, 516–520 (1992).

11. S. Tominaga, ‘‘Multichannel vision system for estimatingsurface and illumination functions,’’ J. Opt. Soc. Am. A 13,2163–2173 (1996).

12. S. Baronti, A. Casini, F. Lotti, and S. Porcinai, ‘‘Multispec-tral imaging system for the mapping of pigments in worksof art by use of principal-component analysis,’’ Appl. Opt.37, 1299–1309 (1998).

13. H. Haneishi, T. Hasegawa, N. Tsumura, and Y. Miyake,‘‘Design of color filters for recording artworks,’’ in Proceed-ings of IS&T’s 50th Annual Conference (The Society for Im-aging Science and Technology, Springfield, Va., 1997), pp.369–372.

14. R. Lenz, M. Osterberg, J. Hiltunen, T. Jaaskelainen, and J.Parkkinen, ‘‘Unsupervised filtering of color spectra,’’ J. Opt.Soc. Am. A 13, 1315–1324 (1996).

15. Munsell Book of Color-Matte Finish Collection (MunsellColor, Baltimore, Md., 1976).

16. T. Jaaskelainen, J. Parkkinen, and S. Toyooka, ‘‘Vector-subspace model for color representation,’’ J. Opt. Soc. Am. A7, 725–730 (1990).

17. T. Jaaskelainen, S. Toyooka, S. Izawa, and H. Kadono,‘‘Color classification by vector subspace method and its op-tical implementation using liquid crystal spatial lightmodulator,’’ Opt. Commun. 89, 23–29 (1992).

18. N. Hayasaka, S. Toyooka, and T. Jaaskelainen, ‘‘Iterativefeedback method to make a spatial filter on a liquid crystalspatial light modulator for 2D spectroscopic pattern recog-nition,’’ Opt. Commun. 119, 643–651 (1995).

19. M. Hauta-Kasari, W. Wang, S. Toyooka, J. Parkkinen, andR. Lenz, ‘‘Unsupervised filtering of Munsell spectra,’’in Proceedings of the 3rd Asian Conference on ComputerVision, ACCV’98, Hong Kong, January 8–10, Vol. 1351


of Lecture Notes in Computer Science, R. Chin andT.-C. Pong eds. (Springer-Verlag, Berlin, 1998), pp. 248–255.

20. E. Oja, Subspace Methods of Pattern Recognition (ResearchStudies Press, Letchworth, UK, 1983).

21. S. Grossberg, Studies of the Mind and Brain (Reidel, Dor-drecht, The Netherlands, 1982).

22. T. Kohonen, The Self-Organizing Maps (Springer-Verlag,Berlin, 1995).

23. S. Haykin, Neural Networks (Macmillan, New York, 1994).24. P. L. Vora, J. E. Farrell, J. D. Tietz, and D. H. Brainard,

‘‘Linear models for digital cameras,’’ in Proceedings ofIS&T’s 50th Annual Conference (The Society for ImagingScience and Technology, Springfield, Va., 1997), pp. 369–372.

25. G. J. Klinker, S. A. Shafer, and T. Kanade, ‘‘The measure-ment of highlights in color images,’’ Int. J. Comput. Vision2, 7–25 (1988).

Documents

Spectral vision system for measuring color images