Spectral multiplexing method for digitalsnapshot spectral imaging
Michael A. Golub,1,* Menachem Nathan,1 Amir Averbuch,2 Eitan Lavi,2
Valery A. Zheludev,2 and Alon Schclar2
1Department of Physical Electronics, Faculty of Engineering, Tel Aviv University, Ramat Aviv 69978, Israel2School of Computer Science, Faculty of Exact Sciences, Tel Aviv University, Ramat Aviv 69978, Israel
*Corresponding author: firstname.lastname@example.org
Received 27 October 2008; revised 5 February 2009; accepted 6 February 2009;posted 13 February 2009 (Doc. ID 103259); published 4 March 2009
We propose a spectral imaging method for piecewise macropixel objects, which allows a regular digitalcamera to be converted into a digital snapshot spectral imager by equipping the camera with only adisperser and a demultiplexing algorithm. The method exploits a multiplexed spectrum intensity pat-tern, i.e., the superposition of spectra from adjacent different image points, formed on the imagesensor of the digital camera. The spatial image resolution is restricted to a macropixel level in orderto acquire both spectral and spatial data (i.e., an entire spectral cube) in a single snapshot. Resultsof laboratory experiments with a special macropixel object image, composed of small, spatially uniformsquares, provide to our knowledge a first verification of the proposed spectral imaging method. 2009Optical Society of America
OCIS codes: 110.4234, 300.6190, 260.2030.
Spectral imaging (SI) provides a two-dimensional(2D) image of a polychromatic object or scene, sepa-rately in several bands of the spectrum . Mosaicspectral filter arrays on an image sensor  leadto unacceptable light gathering losses and involvetechnological difficulties. The most straightforwardway to perform SI is to use a removable set of narrowbandpass filters  or dynamic spectral filters .High quality SI systems exploit staring or push-broom imagers . Neither provides a snapshotSI mode for fast changing objects with an unpredict-able development along the time scale, where theentire spectral cube needs to be acquired simulta-neously and instantaneously in one shot. Snapshotspectral imagers are usually based either on non-scanning computed tomographic imaging spectro-meter (CTIS) designs  or on coded aperturespectral imager designs [13,14] and require a field
stop and additional intermediate image formationoptics.
A wide class of SI applications naturally involvesobjects with a piecewise structure composed of a setof macropixels, all of which have the same uniformor another repeatable local spatial intensity distribu-tion, whose wavelength spectrum varies from macro-pixel to macropixel. Examples of macropixel objectsinclude images of reticle and wafer inspection equip-ment in the semiconductor industry, digitally printedpatterns in the printing industry, and microwellarray plates  and biochip microarrays [16,17]in chemistry and biology.
In this paper we show that snapshot spectral ima-ging of a piecewise macropixel object may beachieved by an exchange between spatial and spec-tral resolution using a regular digital camera. Forthis purpose, the digital camera needs to be equippedwith a disperser whose entire chromatic dispersionmatches the spatial extent of a single macropixel.We show theoretically and experimentally that theentire spectral cube may be obtained by applying di-gital demultiplexing algorithms to the multiplexed
0003-6935/09/081520-07$15.00/0 2009 Optical Society of America
1520 APPLIED OPTICS / Vol. 48, No. 8 / 10 March 2009
spectrum intensity pattern acquired in a single snap-shot. Results of laboratory experiments with amacropixel object composed of small spatially uni-form squares provide a proof of concept for our pro-posed method.
2. Nondispersed Macropixel Image
The term nondispersed image formed by a regularincoherent imaging system (like a digital camera) iscontrasted below with the term dispersed image ac-quired through an imaging system equipped with achromatic disperser. A nondispersed image isdescribed by an intensity function Ix; of 2DCartesian image coordinates x x; y and the wave-length coordinate in a spectral range extendingfrom min to max.Figure 1 shows a spectral cube and its relation to
the discrete pixelated structure of an image sensor,which may be characterized by a 2D spatial pitch x y and a limited number of pixelsNx Ny. The neces-sity of accessing a required number S of discretespectral (wavelength) bands naturally reduces theamount of spatial data Kx Ky provided by the im-age sensor at every spectral band, by about a factorS. Therefore an area of a minimal spatial feature sizeof the nondispersed image has to be about S timeslarger than the area xy of a single image sensor pix-el. We introduce a rectangular macropixel containingSx Sy pixels with a size x y,
x Sxx; y Syy; SxSy S; 1
defining spatial dimensions of each voxel of the non-dispersed image. Each voxel has a spectral dimen-sion j1 j, where j is a representativewavelength of the spectral band numbered j 1;S.An entire wavelength range max min com-prises a column of S voxels of the spectral cube, inaccordance with the S wavelength bands. Equa-tion (1) indicates that the S voxels in a column ofthe spectral cube correspond to Sx Sy S pixelsat the image sensor, to keep a match between the to-tal number of voxels acquired in the SI shot and thetotal number of image sensor pixels.
Each voxel is prescribed with a single value of thenondispersed image intensity V jk;n at discrete rasterpoints k;n k;n; k;n, where k 1;Ky, n 1;Kx,and j 1;S. The nondispersed image of the macro-pixel object with local macropixel intensity distribu-tion gx gx;y is described by
Ix; j XKyk1
Vjk;n Ik;n; j: 2
3. Spectrum Multiplexing Model
Figure 2 shows the optical scheme of a SI imagingsystem that includes a camera lens L, an optionalbandpass filter for the spectral range from minto max, a chromatic disperser G (prism or diffractiongrating) positioned at the pupil, and an image sensorarray D positioned at a focused image plane. The im-aged object is a macropixel object composed of macro-pixels such as to form a macropixel image asdescribed by Eq. (2).
In general, the direction d (jdj 1) of the chromaticdispersion may not coincide with discrete raster di-rections. The disperser shifts laterally every singlepoint image, relative to a nondispersed single pointimage position, by an amount that is assumedto depend only on the wavelength within a fieldof view. An entire dispersive shift jmax
Fig. 1. Relation between (a) the spectral cube data and (b) theimage on the sensor.
Fig. 2. (Color online) Optical arrangement of the experiment forspectral imaging with a prism disperser.
10 March 2009 / Vol. 48, No. 8 / APPLIED OPTICS 1521
minj provides a straight-line rainbow of a dis-persed point image.The entire dispersed polychromatic image at the
image sensor plane has a multiplexed spectrum in-tensity in the sense that data of different voxels arefound at several adjacent spatial locations of the im-age plane. A monochromatic mode image sensorproduces at a point x a signal proportional to the in-tegral of the dispersed image intensity over all wave-lengths
Ix d; d
jIx jd; j; 3
where is a spectral response of the disperser,which in the case of a prism is just the transmittancecoefficient and in the case of a diffraction grating isthe diffraction efficiency of a performing diffractionorder, is the spectral sensitivity of the image sen-sor, is the combined spectral responseof the dispersive element and the image sensor, andj jj1 j is a sampled version of d.Combining Eqs. (2) and (3) yields a multiplexedspectrum image of the macropixel object
jV jk;ngx k;n jd: 4
Note that function Jx of Eq. (4) may be expressedas a 2D spatial convolution J g of the localmacropixel intensity distribution gx and of amultichannel spectrometer image x composedof nonoverlapping point sources x k;n jd,with intensity jVjk;n directly providing readingsproportional to the spectral cube data V jk;n in appro-priately arranged spatial positions k;n jd.Whereas function x and spectral cube data V jk;nare not directly available in an experiment, an ap-proximating function x may be restored fromthe single snapshot Jx by a demultiplexing proce-dure exploiting an appropriate deconvolution algo-rithm, under necessary restrictions on the signal-to-noise (SNR) ratio. This means that equipping adigital imaging system (camera) with a disperserand digital demultiplexing software indeed convertsit into a snapshot SI system for macropixel images.
4. Row-wise Multiplexing and DemultiplexingProcedure
In a row-wise case in which the dispersers direc-tion d of the chromatic dispersion coincides with axisx at the image sensor plane, macropixels extend onlyin a row direction, and there is no interaction be-
tween rows. Accordingly, for each row number k 1;Ky we can exploit one-dimensional (1D) notationsby extracting a single row of the multiplexed spec-trum 2D image. Figure 3 shows the formation ofthe row-wise multiplexed spectrum image withmacropixels in a simplified case of only three spectralbands (S 3) marked 1, 2, 3, where S 3 is chosenin this figure only for graphical purposes and is notlimiting.
The disperser shifts the light of each spectral bandin every macropixel by one detector pixel relative toan adjacent spectral band, as described by the multi-plexed spectrum image equation
Fig. 3. (Color online) Formation of a row-wise multiplexed spec-trum image with macropixels in a simplified case of only threespectral bands (S 3)