Cross talk elimination using an aperture for recording elemental images of integral photography

1RroaAc(tdrtli

tfd

cfs

tnttn

680 J. Opt. Soc. Am. A/Vol. 26, No. 3 /March 2009 Yamamoto et al.

Cross talk elimination using an aperturefor recording elemental images of

integral photography

Kenji Yamamoto,* Tomoyuki Mishina, Ryutaro Oi, Takanori Senoh, and Makoto Okui

National Institute of Information and Communications Technology, 4-2-1 Nukui-Kitamachi, Koganei,Tokyo, 184-8795, Japan

*Corresponding author: [email protected]

Received October 17, 2008; accepted December 19, 2008;posted January 13, 2009 (Doc. ID 102446); published February 26, 2009

A major problem with integral photography using a lens array is overlapping recordings (cross talk) betweenelemental images. Another problem is the decrease in the number of pixels in the elemental images. We de-scribe two methods (including analyses) of manipulating the aperture of a telecentric optical system to improvethese problems. The first method locates the aperture on the focal plane of a field lens. The advantage of thismethod is that cross talk can be reduced without changing the size of the whole optical system. The secondmethod establishes a telecentric optical system between objects and the lens array. The advantage of thismethod, even though the whole optical system becomes bigger, is that cross talk can be completely eliminated.In addition, the number of pixels in the elemental images can be increased by varying the aperture positionsequentially with respect to time. We also describe how cross talk is reduced in both methods by taking dif-fraction into consideration. Experimental results are presented to verify this reduction. © 2009 Optical Soci-ety of America

OCIS codes: 110.6880, 110.1220, 040.7290, 100.6890.

a[pclHsmpbHnm

gias

tSwpanwwriao

. INTRODUCTIONecently, the use of 3D images that can be realisticallyepresented naturally (without special viewing glasses orther equipment) has been investigated in broadcastingnd communications and for archives of cultural assets.lthough many methods [1–5] have been examined for re-ording the objects for these studies, integral photographyIP) capture [1] is considered one of the most promisingechniques among them because light rays in numerousirections can be recorded by a single camera under natu-al light. IP is a 3D image recording and reconstructionechnique using a lens array consisting of numerous tinyenses (elemental lenses), and IP capture is used here tondicate just the recording part of the technique.

However, IP capture has the following problems [5]:

(a) The elemental image created by the light passinghrough the elemental lens is recorded inverted. There-ore, it becomes a “pseudoscopic image” with reversedepth when it is reconstructed.(b) Light that passed through an elemental lens is re-

orded overlapping an adjacent elemental image. There-ore, it becomes a multiple image when it is recon-tructed.

(c) All of the elemental images are recorded in an elec-ronic medium, i.e., in a fixed number of pixels. Since theumber of pixels allocated to each lens is the reciprocal ofhe number of lenses, the number of pixels in an elemen-al image is reduced. Therefore, the field of view becomesarrower.

For (a), it is known that a pseudoscopic image can be

1084-7529/09/030680-11/$15.00 © 2

voided by flipping the image vertically and horizontally6], and this currently is no longer a serious problem torocess in real time due to the improved performance ofomputers. For (b), a method using gradient index (GRIN)enses for the elemental lenses has been proposed [5].owever, manufacturing a GRIN lens array is not neces-

arily easy and it may not be practical even now. For (c), aethod of moving the lens array position has been pro-

osed [7]. If this method is used the frame rate decreases,ut the number of elemental lenses can be increased.owever, since the lens array moves, this method mayot be practical. Also, as far as we know, there still is noethod for dealing with both (b) and (c).Based on the background presented above we investi-

ated a method for (b) that can reduce cross talk while us-ng a lens array consisting of ordinary convex lenses. Welso investigated measures for dealing with (c) at theame time.

In this paper we propose two methods of reducing crossalk. First, we present a brief explanation of cross talk inection 2. Then, in Section 3, we describe technique 1,hich is a method of placing an aperture on the focallane of a field lens located behind the lens array [8]. Thedvantage of this method, even though the cross talk can-ot be completely eliminated, is that the scale of thehole optical system does not change very much. We useave optics here to clearly demonstrate that cross talk is

educed. An optical system similar to the optical systemn Section 3 is introduced in [9]. In [9] the authors placenother lens behind the field lens to form the telecentricptical system behind the lens array and take into consid-

009 Optical Society of America

epittwt

ooetbwcbvttlurteS

2ICIdtpfilr

thtlcaaw

taecfwpcttttt

Sd

t(

3ATLAAralctwtlr

Yamamoto et al. Vol. 26, No. 3 /March 2009/J. Opt. Soc. Am. A 681

ration light beams that pass through the front focaloint of the elemental lens. Section 3 includes the follow-ng: (1) it takes into consideration all light that passeshrough the elemental lens, (2) it investigates the size ofhe aperture, and (3) it contains an analysis based onave optics and shows that this method cannot eliminate

he cross talk completely [8].In Section 4 we describe technique 2, which is a method

f establishing a telecentric optical system between thebjects and lens array. The advantage of this method,ven though the whole optical system becomes biggerhan for technique 1, is that cross talk can theoreticallye completely eliminated. In this section, as in Section 3,e use wave optics to clearly demonstrate that cross talk

an be eliminated. In Section 5 we describe how the num-er of pixels in the elemental images can be increased byarying the aperture position sequentially with respect toime. The ordinary reason to use an aperture in a telecen-ric system is to adjust the brightness of the emergingight and make it parallel. However, in Sections 3–5, wese an aperture to control the direction of the light to beecorded. This differs from the ordinary method of using aelecentric system. Finally, in Sections 6 and 7 we showxperimental results that verified both techniques, and inection 8 we present conclusions.

. CROSS TALK BETWEEN ELEMENTALMAGES IN INTEGRAL PHOTOGRAPHYAPTURE

P capture records an object by placing a recording me-ium behind a lens array consisting of numerous elemen-ary lenses. We assume here that the recording medium islaced on the back focal plane of the lens array. This con-guration, which enables a spatial image with fine reso-

ution to be recorded with a wide range in the depth di-ection, is the arrangement that is generally used [10].

An image (elemental image) of the object viewed fromhe principal point of each elemental lens is projected be-ind each elemental lens onto the recording medium. Athis time, depending on the position of the object, incidentight that enters an elemental lens from that object is re-orded on the neighboring elemental image and becomescross talk component as shown in Fig. 1. Figure 1 showssituation in which light from object B forms an imageithin elemental image 1 (solid triangle in the figure) due

E l e m e n t a l

i m a g e 1

E l e m e n t a l l e n s 1

E l e m e n t a l l e n s 2

E l e m e n t a l

i m a g e 2

b j e c t A

O b j e c t B

R e c o r d i n g m e d i u mL e n s a r r a y

φ

θ

f

D

Fig. 1. Cross talk.

o elemental lens 2. Since this cross talk component islso reconstructed by elemental lens 1, not just by el-mental lens 2, unintended light is reconstructed, whichauses degradation of the reconstructed 3D image. There-ore, it is important for IP capture to record the objectithout causing a cross talk component. A cross talk com-onent occurs when the absolute value of the angle of in-idence � (angle relative to the optical axis of the lens) ofhe incident light that enters an elemental lens exceedshe angle � determined from the focal length f and aper-ure D of the elemental lens. Therefore, to prevent a crossalk component from occurring, the condition indicated byhe following expression must be satisfied [6]:

�� . �1�

ince the following equation holds if f�D (paraxial con-ition) is assumed in an elemental lens,

� =D

2f, �2�

he following relationship is obtained from Eqs. (1) and2):

�� D

2f. �3�

. CROSS TALK REDUCTION TECHNIQUE 1:RRANGEMENT OF APERTURE ONHE FOCAL PLANE OF THE FIELD LENSOCATED BEHIND THE LENS ARRAY. Optical Systems mentioned in Section 2, the principle of IP capture is toecord elemental images by placing a recording mediumt the focal plane of the lens array. However, when theens array is bigger than the recording medium, an opti-al system is used to perform a reduction projection onhe recording medium. This corresponds, for example, tohen the elemental images are recorded by a camera. At

his time, a component that does not enter the cameraens is produced by light that passes through the lens ar-ay as shown in Fig. 2(a). This nonincident component in-

C a m e r a

E l e m e n t a l I m a g el e m e n t a l L e n s

F i e l d L e n s

F o c a l l e n g t h

2f

A p e r t u r e

1fF o c a l l e n g t h

2f

1f

( a ) ( b )

( c )

Fig. 2. Optical system of technique 1.

calfi

coflaafittfitw

loastrc

BCIwteras

p

sArfltnptwdCneas

wfi

Alle

fiLdFFgLsc

HF


reases as more light passes through elemental lensesround the lens array. Therefore, to cause light from theens array to enter the camera lens efficiently, a field lensor converging the light for capture is inserted as shownn Fig. 2(b) [6].

Figure 2(c) shows an optical system for preventingross talk based on the optical system in Fig. 2(b). In theptical system in Fig. 2(c), a field lens (convex lens withocal length f2), the size of which is at least the size of theens array, is placed on the focal plane of the lens arraynd an aperture of width f2D / f1 centered on the opticalxis of the field lens is placed on the focal plane of theeld lens. A camera is placed immediately after the aper-ure (to the right of the aperture in the figure) to recordhe elemental images. As mentioned earlier, when theeld lens is inserted, the only additional part for thisechnique is the aperture. Therefore, the scale of thehole optical system does not change very much.The principal rays passing through the elemental

enses in this optical system are drawn in Fig. 3. Whennly the principal rays are considered, light for which thebsolute value of the angle of incidence is at least D / �2f1�trikes outside the aperture and is intercepted by this op-ical system. As a result, the angle of incident light thateaches the recording medium is limited to D / �2f1�, andross talk is eliminated.

. Analysis That Takes Diffraction Phenomena intoonsiderationn Subsection 3.A we assumed that only the principal raysere being used and showed that the light passing

hrough the elemental lens in Fig. 3 can be limited. How-ver, the light passing through the lens is actually a set ofays, and it is affected by diffraction. In this section wenalyze the proposed optical system by assuming that aet of rays is being used.

We will investigate the lightwave field that the lightassing through the elemental lens in the optical system

F i e l d l e n s

( F o c a l l e n g t h )

D

A p e r t u r e

E l e m e n t a l l e n s

F o c a l l e n g t h )1f

2f

c

φ

φ

φ

1f

2f

12 f

D≅φ

1

2

22

f

Dff =φ

Fig. 3. Trajectories of light rays in technique 1.

hown in Fig. 3 creates on the focal plane of the field lens.lthough the investigation here is only for the x-axis di-ection, the same investigation results are also applicableor the y-axis direction. Let L1i�x1� and L1o�x1� denote theightwave fields on the incident and emergent planes ofhe elemental lens, respectively; let L2i�x2� and L2o�x2� de-ote the lightwave fields on the incident and emergentlanes of the field lens, respectively; and let L3i�x3� denotehe lightwave field on the focal plane of the field lens. Ife let f1 denote the focal length of the elemental lens, cenote the principal point position, D denote the size, andwx

(constant) and wx / f1 denote the brightness and incli-ation, respectively, of the incident light entering the el-mental lens, then the lightwave fields at the incidentnd emergent planes of the elemental lens are repre-ented by the following equations:

L1i�x1� = Cwxexp�− jk

wx

f1x1��x1 − c

D � , �4�

L1o�x1� = L1i�x1�exp�jk�x1 − c�2

2f1� , �5�

here � represents the rectangle function, which is de-ned as follows:

��x1 − c

D � = �1 if c −D

2� x1 � c +

D

2

0 if x1 � c −D

2, c +

D

2� x1

� . �6�

lso, k=2� /�, where � represents the wavelength of theight, and the exp term in Eq. (5) represents the phase de-ay caused in the lightwave by passing through the el-mental lens.

The lightwave field L2i�x2� on the incident plane of theeld lens is obtained by propagating the lightwave field1o�x1� on the emergent plane of the elemental lens by theistance f1. For the propagation calculation we use theresnel transform as an approximation formula of theresnel–Kirchhoff diffraction formula in the Fresnel re-ion. From Eqs. (4)–(6), L2i�x2� and the lightwave field2o�x2� on the emergent plane of the field lens are repre-ented by the following formulas. However, the constantomponents are omitted here.

L2i�x2� =�−

L1o�x1�exp�− jk�x1 − x2�2

2f1�dx1, �7�

=exp�jkx1

2 − x22

2f1�Sa��D

x2 − c − wx

�f1�

exp�− jkx2 − c − wx

f1c� , �8�

L2o�x2� = L2i�x2�exp�jkx2

2

2f2� . �9�

ere, Sa represents the sinc function. Therefore, from theresnel transform and Eqs. (8) and (9) the lightwave field

Lt

EptpalfppTppts

lt�Fwutt

Dtptidpctwwwitt

4AOTAFtpAfaatFD

laFwlcrlc

lno

Ft


3i�x3� on the focal plane of the field lens is represented byhe following formulas:

L3i�x3� =�−

L2o�x2�exp�− jk�x2 − x3�2

2f2�dx2, �10�

=�−

��x2

f2

f1D�exp�jk

wx

f2x2�

exp�− jk�x2 − x3�2

2−f2

2

f1�dx2. �11�

quation (11) represents the Fresnel transform when alane wave with forward direction −wx / f2, which passeshrough an aperture having width f2D / f1 centered on therincipal point, was propagated a distance of −f2

2 / f1. Itlso shows that this does not depend on the elementalens position c. In the Fresnel region, although the dif-raction pattern varies with the propagation distance, alane wave that passed through an aperture mostlyropagates within the extent of the aperture width [11].herefore, a plane wave with inclination wx / f1 thatasses through the elemental lens is considered to mostlyass within the extent of width f2D / f1 having center posi-ion f2wx / f1 in the focal plane of the field lens. Figure 4hows the relationship between wx and the transit area.

From the above, the transit area width in the fieldens’s focal plane of the plane wave for which the inclina-ion � for passing through the elemental lens is ��D /2f1 (that is, �wx��D /2) becomes 2f2D / f1 as shown inig. 4. This is twice the aperture width f2D / f1 in Fig. 3,hich had assumed that only principal rays were beingsed. By letting this region width be the aperture, most ofhe plane wave having inclination ��D /2f1 can be madeo pass through. However, some light having inclination

0

1

2

2

3

f

Df

1

22

f

Df

1

22

f

Df−

2

3 D

2

D D

1

2

2 f

Df−

1

2

2

3

f

Df−

1

2

f

Df−

1

2

2 f

Df

1

2

f

Df

2

3 D−

2

D−

D−

t r a n s i t a r e a o f t h e r a y i n

3x

xw

2

Dw

x≤

t r a n s i t a r e a o f t h e r a y

ig. 4. (Color online) Transit area of the ray in the focal plane ofhe field lens.

/2f0� ��3D /2f1 (shaded parts in Fig. 4) will passhrough at the same time and become the cross talk com-onent. Also, when the aperture width is set to f2D / f1, al-hough the cross talk component will be reduced becauset becomes light having inclination D /2f1� ��D / f1, theesired light having inclination 0� ��D /2f1 will also beartially intercepted. Therefore, the cross talk componentannot be completely eliminated by an aperture placed onhe focal plane of the field lens. Actually, the apertureidth must be determined by taking at least the regionidth f2D / f1 for which the principal rays pass through,hich was shown in Fig. 3, as a criterion while taking

nto consideration the effect on picture quality of the crossalk component and the effect on picture quality due tohe blocking of the desired light.

. CROSS TALK REDUCTION TECHNIQUE 2:RRANGEMENT OF THE TELECENTRICPTICAL SYSTEM BETWEEN OBJECTS ANDHE LENS ARRAY. Optical Systemigure 5 shows the optical system proposed in this sec-

ion. In this optical system, lens L2 (focal length f2) islaced on the plane that is farthest to the front; aperture3, which is centered on the optical axis, is placed on the

ocal plane of L2; and lens L4 (focal length f4) is placed atposition that is further separated from A3 by f4. This isso-called telecentric optical system. The remainder of

he optical system following this, which is the same as inig. 2(b), consists of a lens array L5 (focal length f5, size) and field lens.The principal rays passing through the elemental

enses in this optical system when f4= f2 and the size a ofperture A3 is a= f4D / f5 are drawn using dashed lines inig. 6. When the principal rays are considered, light forhich the absolute value of the angle of incidence is at

east D / �2f5� strikes outside aperture A3 and is inter-epted by this optical system before reaching the lens ar-ay. As a result, the angle of incident light that enters theens array is limited, and cross talk due to principal raysan be considered to be eliminated.

On the other hand, the solid lines and dashed-dottedines in the same figure represent the light rays that areot principal rays of lens array L5. These light rays passr do not pass through aperture A3 based on the absolute

C a m e r a

f

A p e r t u r e

2f

e n s 2L L e n s 4

L

F o c a ll e n g t h

4f

f

F o c a ll e n g t h

3A

5f

L e n s A r r a y5L

F o c a l l e n g t h

D

Fig. 5. Optical system of technique 2.

vTtribft

BCBtFwbmhtLppbraaLeae

pLewLd


alue of the incident angle with D / �2f5� as the boundary.herefore, cross talk is eliminated. From the above, crossalk is eliminated with all light rays, not just principalays. Note that the size of the object for which the images to be reconstructed near the lens array can be changedy setting f2 and f4 to different values. This enables theunction of a depth control lens [5] to be implemented athe same time.

. Analysis That Takes Diffraction Phenomena intoonsiderationy investigating the lightwave field that the light passing

hrough the elemental lens in the optical system shown inig. 7 creates on the focal plane of the elemental lens, weill perform an analysis by assuming that a set of rays iseing used in the proposed optical system in a similaranner as in Subsection 3.B. Although the investigation

ere is only for the x-axis direction, the same investiga-ion results are also applicable for the y-axis direction.et G1o�x1� denote the lightwave field of the front focallane of lens L2 (focal length f2), which is placed at thelane that is farthest to the front, as the lightwave field toe input as shown in Fig. 7. We define G1o�x1� as the rep-esentation of the incident lightwave field to simplifynalysis. However, this definition does not lose generalitys mentioned at the end of this section. Let L2i�x2� and2o�x2� denote the lightwave fields at the incident andmergent planes of the lens L2, respectively. Let L3i�x3�nd L3o�x3� denote the lightwave fields at the incident andmergent planes, respectively, of aperture A , which is

L e n s

F o c a l l e n g t h2

L

42ff =

φ

φ

φ

φ

φ

φ

Fig. 6. Trajectories o

3

laced at a distance of f2 behind lens L2. Let L4i�x4� and4o�x4� denote the lightwave fields at the incident andmergent planes, respectively, of lens L4 (focal length f4),hich is placed at a distance of f4 behind aperture A3. Let5i�x5� and L5o�x5� denote the lightwave fields at the inci-ent and emergent planes, respectively, of lens array L5

Fig. 7. Notations for Subsection 4.B.

A p e r t u r e

3A

L e n s F o c a l l e n g t h

4L

4f

L e n s a r r a yF o c a l l e n g t h

5L

5f

D

54/ fDf=

4f

52/ fD=φ

φ

rays in technique 2.

42ff =

a

f light

(lnlTcl

Gs

Hfi

S

alwalts

tlt

G

ctttffiblo

5ESIwtct(Nhn

tcalda

tGs


focal length f5), which is placed at a distance of l4 behindens L4. This distance is an arbitrary distance l4 that isot restricted to the focal length f4. Let G6i�x6� denote the

ightwave field of the back focal plane of lens array L5.he camera will record G6i�x6�. Also, let c denote the prin-ipal point position and D denote the size of an elementalens constituting the lens array.

We can represent G6i�x6� by using the lightwave field1o�x1� that is input as shown below. However, all con-

tant components are also omitted in this section.

G6i�x6� = exp�jk�l4 − f4 − f5�x6

2 − 2c�l4 − f4�x6

2f52 �

� x6 − c

f5

f4a �F�G1o��kf4/f2f5��x6−c�. �12�

ere, F represents the Fourier transform, which is de-ned as follows:

F�f�x��y =�−

f�x�exp�jxy�dx. �13�

ee Appendix A for the derivation procedure for Eq. (12).From Eq. (12), we know that light that passed through

perture A3 having width a at the common focal plane ofenses L2 and L4 will pass through an extent havingidth f5a / f4 at the back focal plane of lens array L5. Welso know that c is the center of the position where thatight passes through. In other words, if we establish theelecentric optical system before the lens array and let theize a of its aperture be denoted as follows,

a =f4

f5D, �14�

hen, since the light that passes through the elementalens is limited to width D on the recording medium, crossalk between elemental images can be avoided.

In the above analysis the position of the lightwave field1o�x1� that was to be input was restricted to the front fo-

A p e r t u r eL e n s 2L Aφφ

φφ

( a )

Fig. 8. A

al plane of lens L2, which was placed at the plane far-hest to the front. This was done to simplify Eq. (12) andhe equations in its derivation procedure. If we considerhat a lightwave field at an arbitrary position of a plane inront of lens L2 can be transformed by a Fresnel trans-orm to a lightwave field at the position of G1o�x1�, then its obvious that the analysis result stating that cross talketween elemental images can be avoided also holds for aightwave field at an arbitrary position of a plane in frontf lens L2.

. INCREASING THE PIXEL COUNT OFLEMENTAL IMAGES BY APERTUREWITCHING

n Section 4 the center of the aperture had been alignedith the optical axis. As a result, the direction of the light

hat could be captured was ±� (the incident angle is 2�)entered on the optical axis. In this section we considerhat the size a of the aperture remains the same as Eq.14) in technique 2 and the aperture position is moved.ote that although we are only investigating technique 2ere, similar analysis results are also applicable for tech-ique 1.If the center of the aperture is moved by b from the op-

ical axis as shown in Fig. 8(a), the aperture will pass in-ident light for which the center of its direction is not par-llel to the optical axis. In other words, if we define theightwave field in a similar manner as in Section 4 andenote the lightwave field L3o�x3� of the emergent plane ofperture A3 as follows:

L3o�x3� = ��x3 − b

a �L3i�x3�, �15�

hen we can represent G6i�x6� by using the lightwave field1o�x1� that is input as shown below. However, all con-

tant components are omitted.

b

S

T

A p e r t u r e 3A

[ p i e c e s ]

[ p i e c e s ]

a

a

( b )

location.

a

3

perture

Tet−ttDmti

cecbastt

Iwsefh

Uwtfitts

6RAW1mnsntwltnvupiwt

le�Iaceca

BBowtww1lwerttoAh


G6i�x6� = exp�jk�l4 − f4 − f5�x6

2 − 2c�l4 − f4�x6

2f52 �

� x6 − c − f5b/f4

f5

f4a �F�G1o��kf4/f2f5��x6−c�. �16�

he important point to notice here is that the only differ-nce between Eqs. (16) and (12) and is the rectangle func-ion. This difference indicates that the position moves byf5b / f4 (that is, the direction of the light that can be cap-ured changes) and that since a does not change, the lighthat passes through the elemental lens is limited to width

on the recording medium and no cross talk occurs. Byoving the aperture in this way, we can change the direc-

ion of the light that can be obtained while still not caus-ng cross talk.

If we switch the aperture rather than move it, we canreate a formula for the increase in the pixel count of thelemental images. When the image capture device is a TVamera and we let A �pixels�B (pixels) denote the num-er of pixels, IJ denote the number of elemental im-ges, N �pixels�M (pixels) denote the elemental imageize, and F (fps, frames per second) denote the frame rate,hen elemental images of the following size could conven-ionally be captured at F (fps):

N = A/I, �17�

M = B/J. �18�

f we use the proposed technique and consider the casehen the aperture is switched by S �pieces�T (pieces) as

hown in Fig. 8(b), for example (that is, the case when onelemental image is generated by ST frames), although therame rate drops to F /ST (fps), elemental images with aigher pixel count can be captured as follows:

N = AS/I, �19�

M = BT/J. �20�

sing the proposed technique enables elemental imagesith a wider angle of view to be captured than when the

echnique is not used. Therefore, 3D images with a wideeld of view can be captured. In terms of implementation,his method may be practical because a switching aper-ure can be manufactured by using an electrical device,uch as a liquid crystal shutter.

1P

2L

3A

4L

2 0 0 2 0 0 2 0 0 2 0 0

f = 2 0 0 f = 2 0 0a p e r t u r e = 1 2

Fig. 9. Experime

. EXPERIMENT 1: VERIFICATION OF THEEDUCTION OF CROSS TALK. Experimental Setupe verified that cross talk could be reduced by techniquesand 2. Figure 9 shows the configuration of the experi-ental equipment that was prepared to verify both tech-iques under the same conditions, and Table 1 lists thepecifications. Aperture A3, which is the aperture for tech-ique 2, together with lenses L2 and L4 constitute theelecentric optical system for technique 2. Aperture A8,hich is the aperture for technique 1, has been estab-

ished on the focal plane of field lens L7. These two aper-ures are not used at the same time. When verifying tech-ique 1 we removed A3 from the equipment, and whenerifying technique 2, we removed A8. For object P1, wesed characters from a plane on which part of a newspa-er was printed in negative (black and white reversed). P5ndicates the position at which the image was formedhen object P1 was shifted by the telecentric optical sys-

em when aperture A3 was not present.Lens array L6 consists of square (convex) elemental

enses 1.6 mm per side with a focal length of 26.3 mm. Ap-rture A3 is a square aperture 12 mm per side=200 mm1.6 mm/2.6 mm� centered on the optical axis.ts size is shown by Eq. (14). Also, aperture A8 is a squareperture 18 mm per side �=300 mm1.6 mm/2.6 mm�entered on the optical axis. This corresponds to the ap-rture width f2D / f1 that was assumed in Fig. 3. A digitalamera was established immediately after the apertures the capture camera.

. Experimental Resultsy simulating the IP reconstruction system from the setf elemental images captured by the experimental setup,e obtained an image of the object position and verified

he occurrence of a multiple image. First, as in the case inhich cross talk was not reduced, we captured the imageith apertures A3 and A8 in Fig. 9 both removed. Figure0(a) shows the captured image. The space between theines, which should essentially be black, is noticeablyhitened. This is because light that passed through an el-mental lens that is not the relevant elemental lenseached the light receiving element of the part betweenhe lines. In other words, this is the part in which crossalk between elemental images is particularly conspicu-us. Next, to verify technique 1 we inserted only aperture8 to capture the image. In this case, although the objectas been placed at P1, it can be considered equivalent to it

6L

7L

8A

D i g i t a l C a m e r a

[ m m ]

2 0 0 1 2 0 3 0 0

f = 2 6 . 3 f = 3 0 0 a p e r t u r e = 1 8

tup in Section 6.

5P

ntal se

battcw

r1mfihputn


eing placed at P5. Figure 10(b) shows the captured im-ge. Finally, to verify technique 2 we inserted only aper-ure A3 to capture the image. Figure 10(c) shows the cap-ured image. It is apparent that cross talk is lessonspicuous in Figs 10(b) and 10(c) and than in 10(a),hich was captured when cross talk was not reduced.Figures 10(d)–10(f), are images of the object that were

econstructed from Figs. 10(a)–10(c), respectively. Figure0(d) is the result when cross talk was not reduced. Aultiple image apparently occurred, and the object is dif-cult to recognize. Also, since the space between the linesas been whitened, the contrast between the characterarts and space between the lines parts is degraded. Fig-re 10(e) is the result when technique 1 was used. Al-hough the multiple image has not been completely elimi-ated, it has become less conspicuous, and the object can

expansion

( b )

( d )

( f )

results for reducing cross talk.

Table 1. Specifications of Experimental Setup inSection 6

ItemValue(mm)

Focal length of telecentric optical system’slenses L2 and L3

200

Aperture size of aperture A3 12�H�12�V�Focal length f6 of elemental lens of lens

array L6

26.3

Aperture D of elemental lens of lensarray L6

1.6�H�1.6�V�

Focal length f7 of field lens L7 300Aperture size of aperture A8 18�H�18�V�

expansion

expansion

( a )

( c )

( e )

Fig. 10. (Color online) Experimental

bamsm1rwFsiboe

7ITForesstsepT

etei

acc

psjm1iada

tpd

8TeewttcWttritetpp

ACTSbtta


e recognized. We think that the remaining multiple im-ge can be further reduced by adjustments such as byaking the aperture size smaller. Figure 10(f) is the re-

ult when technique 2 was used. It is apparent that theultiple image has been eliminated more than in Fig.

0(e). In particular, there is a noticeable difference in theesult for the space between the lines’s parts comparedith Fig. 10(e). These experimental results, shown inigs. 10(d)–10(f), qualitatively match the analysis de-cribed in the earlier sections. The blurring of all of themages in Figs. 10(d)–10(f), can be considered to be causedy the distortion of the optical system such as distortionr aberration of the lenses or error in the placement of ap-rtures or lenses.

. EXPERIMENT 2: VERIFICATION OF THENCREASE IN THE PIXEL COUNT OFHE ELEMENTAL IMAGESirst, we verified whether the ray direction that can bebtained by aperture switching varies. The experimentecorded the object while switching the position of the ap-rture of the telecentric optical system so that there wasome degree of overlap rather than it being in the tilehape shown in Fig. 8(b). We did this so that combininghem would be easier by using the overlap. Figure 11hows the experimental setup. Since we only used the el-mental lenses of the lens array that were on the frontlane of the digital camera lens, no field lens was used.he object was part of a newspaper.We captured numerous images while switching the ap-

rture. Figures 12(a) and 12(b) show examples of cap-ured images. Images that passed through the same el-mental lens are surrounded by a solid and a dashed linen the figure. Since the images surrounded by the solid

1P

2L

3A

4L

5L

D i g i t a l C a m e r a

[ m m ]

7 0 0 1 5 0

f = 2 6 . 3f = 1 5 0a p e . = 9f = 1 5 0

1 5 0

Fig. 11. Experimental setup in Section 7.

( b )( a )

Fig. 12. (Color online) Experime

nd dashed lines differ, it is apparent that the imagesould be captured while varying the ray direction thatould be obtained.

Next, we verified that elemental images having highixel counts could be produced. Figure 12(c) shows the re-ult when one elemental image was created by combiningust the images that passed through the elemental lens

entioned earlier. In Fig. 12(c) the parts used in Figs.2(a) and 12(b) are surrounded by solid and dashed linesn the figure. It is apparent that images that convention-lly could only capture the portion inside the solid orashed lines can be combined in an elemental image withwide angle of view and an increased pixel count.Although we only showed the result here for images

hat passed through one elemental lens, images thatassed through the other elemental lens can also be pro-uced in a similar manner.

. CONCLUSIONSwo major problems in IP capture are cross talk betweenlemental images and the decrease in the pixel count oflemental images. We proposed two methods for dealingith both problems. One method places an aperture on

he focal plane of a field lens located immediately afterhe lens array, and the other method establishes a tele-entric optical system between the objects and lens array.e used wave optics to clearly demonstrate that cross

alk between elemental images is reduced by using thesewo methods. We also described how we can vary the di-ection of light ray information that can be obtained andncrease the pixel count of elemental images by switchinghe aperture position. In addition, we performed opticalxperiments to verify these results. In the future, we plano develop a real-time holographic movie system for dis-laying 3D images by capturing images using the pro-osed techniques and converting them to holograms.

PPENDIX A: DERIVATION OF EQ. (12) INROSS TALK REDUCTIONECHNIQUE 2ince L2o�x2� is affected by the propagation of lightwavesy a distance of f2 and the phase delay due to passinghrough lens L2 relative to G1o�x1�, it is represented byhe following formula. However, all constant componentsre omitted in this section.

( c )

esults for high resolution image.
ntal r

wtLE

wlr

L

wdotf

G

Rat

R


L2o�x2� = exp�jkx2

2

2f2�L2i, �A1�

=exp�jkx2

2

2f2��

−

G1o

exp�− jk�x1 − x2�2

2f2�dx1, �A2�

=F�G1o exp�jk− x1

2

2f2��

�k/f2�x2

. �A3�

Since L3o�x3� is affected by the propagation of light-aves between lens L2 and aperture A3 and the restric-

ion of the transit area due to aperture A3 relative to2o�x2�, it is represented by the following formula fromqs. (A1)–(A3):

L3o�x3� = �x3

a L3i, �A4�

=�x3

a �−

L2o exp�− jk�x2 − x3�2

2f2�dx2, �A5�

=�x3

a �−

F�G1o exp�jk− x1

2

2f2��

�k/f2�x2

exp�− jk�x2 − x3�2

2f2�dx2, �A6�

=�x3

a F�G1o��k/f2�x3. �A7�

Since L4o�x4� is affected by the propagation of light-aves between aperture A3 and lens L4 and the phase de-

ay due to passing through lens L4 relative to L3o�x3�, it isepresented by the following formula from Eq. (A7):

4o�x4� = exp�jkx4

2

2f4�L4i, �A8�

=exp�jkx4

2

2f4��

−


2f4�dx3, �A9�

=exp�jkx4

2

2f4��

−

�x3

a F�G1o��k/f2�x3

exp�− jk�x3 − x4�2

2f4�dx3, �A10�

=F��x3

a F�G1o��k/f2�x3exp�jk

− x32

2f4��

�k/f4�x4

.

�A11�

Since G6i�x6� is affected by the propagation of light-aves between lens L4 and lens array L5, the phase delayue to passing through lens array L5, and the propagationf lightwaves by a distance of f5 from lens array L5 rela-ive to L4o�x4�, it is represented by the following formularom Eq. (A11). Equation (A18) is the same as Eq. (12):

6i�x6� =�−


2f5�dx5, �A12�

=�−

exp�jk�x5 − c�2

2f5�L5i exp�− jk

�x5 − x6�2

2f5�dx5,

�A13�

=exp�jk− x6

2

2f5�F�L5i��k/f5��x6−c�, �A14�

=exp�jk− x6

2

2f5�F��

−

L4o

exp�− jk�x4 − x5�2

2l4�dx4�

�k/f5��x6−c�

, �A15�

=exp�jk�l4 − f5�x6

2 − 2cl4x6

2f52 �F�L4o�−�k/f5��x6−c�,

�A16�

=exp�jk�l4 − f5�x6

2 − 2cl4x6

2f52 �

F�F��x3

a F�G1o��k/f2�x3

exp�jk− x3

2

2f4��

�k/f4�x4

�−�k/f5��x6−c�

, �A17�

=exp�jk�l4 − f4 − f5�x6

2 − 2c�l4 − f4�x6

2f52 �

� x6 − c

f5

f4a �F�G1o��kf4/f2f5��x6−c�. �A18�

ectangle function � is ignored in Eq. (A18), when theperture A3 is removed; in other words, a is infinite. Crossalk occurs in this case.

EFERENCES1. M. G. Lippmann, “Epreuves reversibles donnant la

sensation du relief,” J. Phys. 7, 821–825 (1908). Englishtranslation available at http://www.tgeorgiev.net/Lippman/index.html.

2. T. Fujii, K. Mori, K. Takeda, K. Mase, M. Tanimoto, and Y.Suenaga, “Multipoint measuring system for video andsound—100-camera and microphone system,” in 2006 IEEE

1

1


International Conference on Multimedia and Expo (IEEE,2006), pp. 437–440.

3. I. Yamaguchi and T. Zhang, “Phase-shifting digitalholography,” Opt. Lett. 22, 1268–1270 (1997).

4. M. Kawakita, K. Iizuka, T. Aida, H. Kikuchi, H. Fujikake,J. Yonai, and K. Takizawa, “Axi-vision camera (real-timedistance-mapping camera),” Appl. Opt. 39, 3931–3939(2000).

5. J. Arai, M. Okui, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integralphotography for three-dimensional image,” Appl. Opt. 37,2031–2045 (1998).

6. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-timepickup method for a three-dimensional image based onintegral photography,” Appl. Opt. 36, 1598–1603 (1997).

7. J. Jang and B. Javidi, “Improved viewing resolution of

three-dimensional integral imaging by use of nonstationarymicro-optics,” Opt. Lett. 27, 324–326 (2002).

8. T. Mishina, K. Yamamoto, R. Oi, and M. Okui, “Reducing ofinterference between elemental images during integralphotography pickup suitable for producing holograms,” J.Inst. Image Inf. Telev. Eng. 62, 1132–1137 (2008) (inJapanese).

9. R. Martinez-Cuenca, A. Pons, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Optically-corrected elementalimages for undistorted integral image display,” Opt.Express 14, 9657–9663 (2006).

0. J. Arai, H. Hoshino, M. Okui, and F. Okano, “Effects offocusing on the resolution characteristics of integralphotography,” J. Opt. Soc. Am. A 20, 996–1004 (2003).

1. E. Hecht, Optics, 4th ed. (Addison Wesley, 2002), pp.485–509.

Documents

Cross talk elimination using an aperture for recording elemental images of integral photography