Design and scaling of monocentric multiscale imagers

Eric J. Tremblay,1,3,* Daniel L. Marks,2 David J. Brady,2 and Joseph E. Ford1

1Department of Electrical & Computer Engineering, University of California San Diego, 9500 GilmanDr., La Jolla, California 92093-0407, USA

2Fitzpatrick Institute for Photonics, Duke University, 101 Science Drive, Durham, North Carolina 27708, USA3Currently with the Institute of Microengineering, EPFL, Station 17 CH-1015 Lausanne, Switzerland

*Corresponding author: eric.tremblay@epfl.ch

Received 29 November 2011; revised 6 May 2012; accepted 8 May 2012;posted 8 May 2012 (Doc. ID 159061); published 4 July 2012

Monocentric multi-scale (MMS) lenses are a new approach to high-resolution wide-angle imaging, wherea monocentric objective lens is shared by an array of identical rotationally symmetric secondary imagersthat each acquire one overlapping segment of a mosaic. This allows gigapixel images to be computation-ally integrated from conventional image sensors and relatively simple optics. Here we describe theMMS design space, introducing constraints on image continuity and uniformity, and show how paraxialsystem analysis can provide both volume scaling and a systematic design methodology for MMS imagers.We provide the detailed design of a 120° field of viewimager (currently under construction) resolving2 gigapixels at 41.5 μrad instantaneous field of view, and demonstrate reasonable agreement withthe first-order scaling calculation. © 2012 Optical Society of AmericaOCIS codes: 220.2740, 110.1758, 080.3620, 040.1240.

1. Introduction

Wide-angle gigapixel images are most commonlyacquired by angle scanning a conventional high-resolution camera to sequentially form a compositeimage of the scene [1]. However, the long acquisi-tion time and time lapse between first and lastframes make this approach only practical for station-ary objects. For some applications such as wide-areapersistent surveillance, it is necessary to acquire aslarge a field of view (FOV) as possible with resolutionto allow for long-range operation without loss of ob-ject details, all within the time frame of a singleacquisition. These requirements easily lead to pixelcounts in the multi-gigapixel range and exceed whatis available as a monolithic focal plane.

While it is possible to tile smaller sensor die into alarger contiguous defect-free chip, this approach isdifficult and costly [2,3]. One approach to avoid amonolithic focal plane is to allow spaces between

the image sensor tiles, and use multiple imagers tofill in the interleaving image data, as in DARPA’sARGUS-IS. Here, four co-boresighted imaging sys-tems are multiplexed together to provide a 1.8 giga-pixel image over a 60° FOV [4]. This is a successfulapproach, but to our knowledge it has not been ex-tended to very large FOV, where optical design chal-lenges become more difficult. This is due to aphysically large flat image plane for which a wide-angle lens design is possible but difficult since geo-metric aberrations scale with lens focal length whilediffraction does not [5]. A straightforward alternativefor very large FOV, high-resolution imaging is themulticamera array, where images are simultaneouslycomposited from an array of independent cameras[6]. However, as we will show, the physical area ofthe optics becomes prohibitive for a multi-camera ar-ray at sufficiently large image sizes.

In 2009, Brady and Hagen proposed a multiscalelens approach, bridging the gap between a single sen-sor imager andmulticamera array [7]. In amultiscaleimager, the lens system is separated into a single pri-mary lens and multiple secondary subimagers. The

1559-128X/12/204691-12$15.00/0© 2012 Optical Society of America

10 July 2012 / Vol. 51, No. 20 / APPLIED OPTICS 4691

objective acts as a common primary optic with sharedaperture, but instead of forming a contiguous imageplane, the system is divided into multiple imagefields by multiple sets of secondary optics, each withits own focal plane. As in a multicamera array, thefinal image is computationally assembled by stitch-ing together the overlapping FOVs recorded on con-ventional image sensors. As in a multicamera array,the image is segmented into smaller focal planes,and the secondary optics need to correct aberrationsover a limited field angle. But in a multiscale ima-ger, the largest optical elements are shared by all ofthe secondary subimagers. In the simplest case, thesecondary camera array forms a virtual focal planewith demagnifying relay optics, which simplifiesthe challenge of large focal plane synthesis fromsmaller arrays [8]. More unique and significant ad-vantages are obtained if individual subimagerscorrect for aberrations of their corresponding off-axis perspective of the primary lens and for localdefocus and exposure requirements. With conven-tional planar focal plane primary lenses, localaberration correction requires a varying array offield-dependent “freeform” aspheric optics, whichhave no rotational symmetry and are difficult tofabricate and align. This challenge is alleviated,however, by noting that lens design for a flat fieldis a legacy constraint of single element focal sur-faces, and the primary lens in a multiscale systemno longer requires an overall flat focal plane.

Taking the curved focal plane to an extreme resultsin a hemispherical focal surface centered at the rearprinciple point of the objective lens. This turns out tobe useful, and leads toward an even more symmetricsystem geometry. The choice of design symmetryshown in Fig. 1 dramatically simplifies the design

and fabrication of wide-field multiscale imagers[9,10]. Making the primary a hemispherically-symmetric monocentric lens [11] means that theaberrations of the primary will be independent offield angle, so that a single on-axis, rotationally sym-metric design can be optimum for all the subimagers.Monocentric lenses are a well-established solution towide angle imaging, with work dating from 1861with Thomas Sutton’s Panoramic Lens, a flint glasssphere filled with water [12] to the present day [13].Recently, building on their concepts for an omnidir-ectional spherical imager [14], Cossairt and Nayerimplemented a closely related configuration usinga glass ball and single element relay lenses. Thisimager recorded overlapping images on 5 adjacentimage sensors, and was demonstrated with anglescanning to acquire and integrate a 1.6 gigapixelimage [15].

Multiscale imagers are intrinsically a computa-tional imaging system, and post-detection processingcan significantly improve the effective resolution.This does not mean, however, that physical resolu-tion of the detected image is unimportant. Whilesome conventional aberrations, e.g., distortion andfield curvature, are easily accounted using computa-tional correction and multiscale design, the impact ofdefocus, chromatic aberration, spherical aberration,coma, and astigmatism cannot be completely re-moved in computation. In practice, reducing theseaberrations minimizes the computational resourcesneeded, limits visual artifacts in the final image,and allows operation under the broadest lightingconditions.

The optical design of monocentric multiscale(MMS) imagers presents some unique challenges.The subimager fields of view must overlap suffi-ciently that a complete mosaic image can be com-puted, with the image resolution maintained intothe overlap area. In the overlap regions, light fromthe primary lens is necessarily divided to illuminateeither two or three adjacent image sensors, and theoptomechanics used to mount the secondary opticalelements tend to introduce dead space betweenadjacent secondary optics apertures. Control of vi-gnetting losses and stray light suppression aretherefore important to maintain an acceptable sig-nal to noise ratio at the subimager field edges. Tomake the impulse response of all subimager fieldsidentical, the aperture stop for the overall systemshould lie within the subimager, and the correctchoice of subimager geometry and aperture can finda balance between image uniformity and overalllight collection efficiency.

In this paper, we define two major classes of MMSimagers, those with and without an internal imageplane, and show how imposing constraints on imagecontinuity and uniformity can narrow the designspace to valid solutions of both types. We showhow a paraxial system analysis can start from basicdesign parameters (IFOV, FOV, sensor size and pixelpitch) to define the overall lens structure, allowing

Fig. 1. (Color online) First-order MMS lens models. (a) Keplerian-type and (b) Galilean-type.

4692 APPLIED OPTICS / Vol. 51, No. 20 / 10 July 2012

volume scaling calculations as well as a systematicdesign methodology for MMS imagers. With first-order design and scaling relations established wethen compare scaling of the MMS lens architectureto that of an equivalent multicamera array. Finally,we illustrate this process with the detailed opticaldesign of a 2 gigapixel 120° FOV imager, which is cur-rently under construction with a large team effortlead by Duke University as part of the DARPAAdvanced Wide FOVArchitectures for Image Recon-struction (AWARE) program [16].

2. First-Order Models

A MMS lens system can be considered as an array ofsubimaging systems that relay with local aberrationcorrection the spherical image plane produced by amonocentric objective to an array of conventional flatfocal planes. A useful conceptual alternative to thisview, which allows for a particularly clean first-orderanalysis, is to consider the system as a telescope ar-ray where the array elements share a common con-centric objective lens. Figure 1 shows the simplifiedlens arrangement for one channel of theMMS systemwith Keplerian and Galilean type arrangements.A Keplerian telescope uses a positive primary lensand positive secondary lens, spaced by the sum oftheir focal lengths, to create an inverted afocal systemwith an angular magnification of the ratio of the pri-mary and secondary lenses. A Galilean telescope usesa negative secondary lens, again spaced by the sum ofthe primary and secondary lens focal lengths, to createa non-inverting afocal telescope with an angular mag-nification of the ratio of the primary and secondarylens powers. In both cases, a lower angular resolutioncamera (lens� focal plane) can be used to record themagnified image.

To create a simple first-ordermodel for the twoMMSarrangements, we can consider a one-dimensional (1D)array of thin lens elements and starting parameters:total system FOV, FOVT , IFOV, wavelength of light,λ, 1D number of pixels per focal plane, NP, and pixelpitch, p.

Considering a 1D array of secondary optics(eyepiece� array camera), the total number of pixelsin 1D is NT � FOVT∕IFOV. With the sensor chosen,the number of 1D array elements is then given byNA � NT∕Np, and the half angle required by eachis α � FOVT∕2NA, where α is the half-FOV per sub-imager not including any image overlap. The focallength of the array camera will be given by

f AC � XS

2jMj tan�α� ; (1)

where M is the angular magnification of the tele-scope, M � −f o∕f e, given by the ratio of the focallengths of the objective and eyepiece and XS is the1D focal plane size: XS � pNp. From Eq. (1), we cansee that the total focal length of the MMS lens sys-tem is fMMS � jMjf AC � jf o∕f ejf AC. To resolve thepixel pitch of the focal plane, we can use the

Rayleigh criterion to specify a maximum F∕# forthe array camera, which is also the F∕# for the over-all system, F∕#AC � p∕1.22λ. Combining this condi-tion with Eq. (1) gives the required aperture of thearray camera,

DAC � 0.61 ·NpλjMj tan�α� : (2)

The packing of the array and the FOV of the arraycamera determine the aperture of the eyepiece. Themaximum eyepiece aperture, mechanically and formost effective light collection, occurs when the eye-piece is in contact with the adjacent eyepiece lensesof the array

DEP � 2d tan�α�; (3)

where DEP is the diameter of the eyepiece, and d is theseparation between the objective and eyepiece as shownin Fig. 1. To determine the remainingmodel parameterswe must consider vignetting within the system andtradeoffs in the particular system geometry.

A. Keplerian-Type Model

In the traditional Keplerian arrangement the stopand entrance pupil are located at the objective, pla-cing the exit pupil at the eye (array camera in ourcase) with eye relief, lrelief � −d∕M [17]. Vignettingand the FOV of the telescope with the stop in the ob-jective are determined by the size of the telescopeeyepiece. With a MMS lens system the eyepiece islimited in size and packing of the adjacent eyepiecelenses of the array, and the frequency of the array isdetermined by the FOV covered by each array cam-era. With the stop in the objective and adjacentarray eyepieces in contact, 50% vignetting occursat the transition between array cameras. Dashedlines in Fig. 1 show the path of the chief ray to theeyepiece at the FOV of the array camera. The onsetof vignetting occurs at a field angle of

αVonset �DEP −DAC

2d: (4)

In addition to the vignetting associated with transi-tioning between array cameras, a MMS lens with thestop in the objective will incur a cos�θ� falloff in re-lative illumination (RI) due to the obliquity of theaperture stop. Figure 2 shows an example first-order

Fig. 2. (Color online) Keplerian-type with stop in objective.(a) Layout showing three subimagers, and (b) relative illuminationfor a full field of �60°.

10 July 2012 / Vol. 51, No. 20 / APPLIED OPTICS 4693

Keplerian design with its associated RI showing thelarge variations in RI between subimagers andacross the full FOV.

Vignetting is traditionally reduced in Kepleriantelescopes with the use of a field lens placed closeto the intermediate image. Field lenses can be addedto the Keplerian-type MMS design to completelyeliminate the vignetting between subimagers pro-vided that there are no gaps, i.e., sharp transitions,between adjacent field lenses. With a field lens theeye relief to place the exit pupil at the array camerais

lrelief ;FL � −dM

−f 2EPf FL

; (5)

where f FL is the focal length of the field lens given by

f FL;max �f 2EP

−dM � DEP−DAC

2M tan α: (6)

The value of f FL in Eq. (6) is a maximum, represent-ing the lowest power and least aberrating field lensto achieve zero vignetting in the MMS camera.

Although field lenses can correct problematic var-iations in RI across the full stitched image, a single-piece, precision hemispherical array of field lenseswith the necessary sharp transitions and tight toler-ances is a difficult and high-risk task with currentfabrication capabilities [18]. An attractive alterna-tive approach to improve vignetting and RI withoutfield lenses is to place the stop at each array camerainstead of the monocentric lens. In terms of the over-all multiscale design this can be thought of as localcontrol rather than global control of the light bundle.With a local stop at the array camera, the objectivelens now appears identical to each subimager andthere is no cos�θ� obliquity factor across the fullFOV. In addition, vignetting between adjacent subi-magers can be reduced to zero as light is allowed tofill the local stop at the array camera. A consequenceof this reduction in vignetting is a departure from themonocentric condition for the objective within eachlocal imaging path. In this way the monocentric con-dition applies globally rather than locally for eachsubimager. A first-order Keplerian-type design ex-ample with the stop at the array camera is shownin Fig. 3. Since there is no geometrical vignetting

the roll-off in RI at the transitions between subima-gers is well approximated by the cosine fourth law ofthe array camera alone [Fig. 3(b)].

For zero vignetting with the stop at the array cam-era, the array camera and eyepiece must be arrangedas shown in Fig. 4(a) with the relationship

tan�α0� � DEP −DAC

2lrelief; (7)

where α0 is the magnified field angle at the arraycamera given by

α0 � tan−1�jMj tan�α��: (8)

Combining with expressions for the eyepiece andarray camera diameters gives the eye relief for zerovignetting:

lrelief �f oM2 −

f oM

−XS

4M2F∕#AC tan2�α� : (9)

With the stop and exit pupil at this location with exitpupil diameter, Dε0 � DAC the entrance pupil will bemoved to the location

lε �MDAC

2 tan�α� ; (10)

in front of the objective with a diameter given byDε � jMjDAC. The diameter of the objective lens re-quired for zero vignetting is found from the locationand size of the entrance pupil to be

Do �XS

F∕#AC tan�α� � 2Dε; (11)

Fig. 3. (Color online) Keplerian-type with stop at the arraycamera. (a) Layout showing three subimagers, and (b) relativeillumination for a full field of �60°.

Fig. 4. (Color online) Detail for calculation of vignetting with stopat the array camera. (a) Keplerian-type at the onset of vignetting,and (b) Galilean-type at the onset of vignetting.

4694 APPLIED OPTICS / Vol. 51, No. 20 / 10 July 2012

which is twice as large as the entrance pupildiameter, Dε.

B. Galilean-Type Model

In a traditional Galilean telescope the eye is the stopand exit pupil of the system. In our case the stop andexit pupil are placed at the array camera (analogousto the eye). With the stop located at the array cam-era, vignetting is determined by the size of objectivelens’s aperture. With a Galilean-type MMS systemwe can first assume that no eye relief is necessary(lrelief � 0) since additional eye relief only increasesvignetting for a given objective lens diameter. Sincethe eyepiece and array camera are collocated, and ul-timately combined, we can also assume that theyhave the same diameter, DEP � DAC, which specifiesa magnification for ideal packing:

Mopt �Xs

4f o tan2�α�1

F∕#AC� 1. (12)

This value of magnification is an optimum since lar-ger values cannot be achieved without reducing DACand smaller values increase wasted space and vignet-ting for a given objective aperture size. In addition,there is a physically constrained minimum for themagnification since the eyepiece must be located out-side the monocentric objective. For example, magnifi-cation will be constrained to approximately Mopt ≥ 2for dual material monocentric designs, and Mopt ≥ 3for single material monocentric designs based on thediameter of the objective lens. With the Galilean-typedesign, the entrance pupil will be located at, lε � Md,behind the objective lenswith apertureDε � MDAC asshown in Fig. 4(b). The field angle where vignettingjust begins is given by

tan�α� � �Do∕2� −M�DEP∕2�Md

: (13)

Rearranging Eq. (13), the diameter of objective lens,Do, for zero vignetting is equivalent to Eq. (11), thevalue for the Keplerian-type design with the stop atthe array camera.

C. Considerations for Two-Dimensional Tiling

The preceding discussion dealt with a 1D tiling forsimplicity. Real designs with two-dimensional (2D)tiling are significantly more complex and require anadequate solution to the Tammes problem to maxi-mize the sphere packing density [19]. One practicalscheme uses a modified icosahedral geodesic toachieve high packing density with a combinationof hexagonal and pentagonal tiling, and irregularspacing over the hemispherical array of subimagers[20,21]. The design parameters given by the 1D tilingequations understate the vignetting and RI of a realsystem due to the larger subimager separations incertain directions, such as those along hexagonalvertices, and irregular packing. Figure 5 illustratesby example possible variations in RI due to field

direction and irregularity. Such variations in RI for2D tiling are significant, but can be estimated basedon the packing scheme and are generally reasonablewhen the zero vignetting 1D equations are used as adesign starting point. When using the modified ico-sahedral geodesic packing scheme, the most area-efficient shape for tiling sensors to cover the 2Dimage field is a hexagon, as shown in Fig. 5. While itis possible to make hexagonal focal planes, approxi-mately 35% of the pixels in a conventional 4∶3 aspectratio sensor would either be redundant and used forimage stitching or not illuminated.

3. System Scaling

A. Monocentric Multiscale Lens Systems

Once the starting parameters have been chosen, theoptical designer must choose values for the free para-meters of the system to fully define the first-ordermodel. For the Keplerian-type there are two freeparameters: magnification (M) and the F∕# of theobjective lens given by

F∕#o �f oDε

� 2f oDo

; (14)

assuming the object at infinity. The Galilean-typehas only one free parameter, F∕#o Eq. (14), sincemagnification is fixed to an optimal value given byEq. (12). F∕#o is practically limited in a MMS lensby the structure and performance of the monocentriclens. It is sometimes suggested that the F∕# of animaging lens tends to be approximately the cube rootof its focal length, F∕#o �

�����f o

3p

[5,22]. While not a con-straint and often inaccurate, this rule of thumb canbe useful for choosing a design starting point. If com-bined with Eq. (11) and Eq. (14), F∕#o is eliminatedas a free parameter. In this discussion of scaling wewill leave F∕#o as a free parameter to examine its ef-fect. In addition, since the aperture of the objective,Do, is required to be 2× the entrance pupil diameterfor zero vignetting, values of F∕#o < 3 are not realis-tic with a crown/flint monocentric triplet (see Sec 4for an example). Smaller values of F∕#o are possiblewith single element ball lenses; however we will con-strain our discussion to achromatized monocentric

Fig. 5. (Color online) (a) Relative illumination for (b) flat-flat fieldorientation with ideal hexagonal packing, and (c) vertex-edge fieldorientation with a 15% spacing irregularity.

10 July 2012 / Vol. 51, No. 20 / APPLIED OPTICS 4695

triplets which are better suited to gigapixel scaleresolutions.

Although magnification is a free parameter in theKeplerian-type design, it has a geometrically con-strained minimum value when the eye-relief Eq. (9)goes to zero:

jMminj �1

2F∕#o tan�α� − 1. (15)

The minimum values of magnification are plotted inFig. 6(a) as a function of 2D resolution where we havechosen a specific sensor geometry (Np � 2800 pixels,with pitch p � 1.4 μm) typical of current commerciallyavailable CMOS image sensors. 2D resolution is esti-mated as NT;2D � π

4N2T, assuming symmetric FOV. In

addition to this geometrical constraint on the mini-mum value of magnification, values of magnificationless than or equal to one (M ≤ 1) have no practical sig-nificance since the system could be replaced with asmaller and simpler multicamera array. The potentialadvantage to the MMS arrangement is linked to thevalue of this magnification and practically requireslarger values.

As previously noted, magnification for theGalilean-type is constrained to be Mopt ≥ 2 by thephysical separation of the monocentric objectiveand the eyepiece lens. Figure 6(b) shows the optimalmagnification of the Galilean-type, given by Eq. (12),as a function of 2D resolution. Note that the optimal

magnification of the Galilean-type and the minimummagnification of the Keplerian-type are simplyrelated: Mopt � jMminj � 2.

The optical volume of Keplerian and Galilean-typeMMS lenses can be calculated as the volume sum of afront hemisphere with the radius of the objective anda larger partial rear hemisphere with a radius total-ing the track length of the rear concentric optics (ob-jective, eyepiece and array camera). The partial rearhemisphere is made up of a cylinder and a sphericalcap (Fig. 6 includes a sketch) [23]. Assuming 120°FOVT , some geometric manipulation gives the totaloptical volume of the system as

Volume � 2.09r3f � 1.8325r3b; (16)

where rf is the front radius of the monocentric lensand rb is the distance from the center of the mono-centric lens to the final image plane. Assuming a spe-cific but commonmonocentric triplet, such as the onedescribed in Sec. 4, we find that the front radius andfocal length are related as rf ≈ 0.475f o For theKeplerian-type design, rb will be given by the sumof parameters d, lrelief and f AC, or

rb;Keplerian � XS

2 tan�α�

�F∕#oF∕#AC

�1 −

1M

�2−

1M

−1

2M2F∕#AC tan�α�

�: (17)

Fig. 6. (Color online) Scaling of first-order model parameters for fixed sensor size (Np � 2800, p � 1.4 μm). (a) Keplerian-type minimumrequired magnification versus 2D resolution, (b) Galilean-type optimal magnification vs. 2D resolution, (c) Keplerian-type optical volumevs. 2D resolution and (d) Galilean-type optical volume vs. 2D resolution. (c) and (d) both use the “optimal” value of magnification from (b)for comparison. Solid line volume curves (c and d) represent more realistic sensor packing requirements due to sensor packaging (ρ � 2).Dashed line volume curve represents the extension to the curve possible when the sensor package is not included (ρ � 1).

4696 APPLIED OPTICS / Vol. 51, No. 20 / 10 July 2012

For the Galilean-type design rb will be given by thesum of parameters d, and f AC or

rb;Galilean � XS

2 tan�α�F∕#oF∕#AC

�1 −

1M

� F∕#ACF∕#oM

�: (18)

Further, sensor packing presents a practical con-straint particularly when the packaging of the imagesensors is taken into account. This constraint can beexpressed by enforcing a value of rb large enough toallow for separation between the image sensors:

rb ≥ρXS

2 tan�α� ; (19)

where ρ is a scaling factor representing the size ofthe image sensor package compared to the activepixel array.

Figures 6(c) and 6(d) show volume scaling asfunction of 2D resolution for the Keplerian-type andGalilean-type designs, respectively, using the samefixed sensor size (Np � 2800, p � 1.4 μm). To makethe charts as comparable as possible, the “optimal”magnification of the Galilean-type Eq. (12) was usedfor both. Comparing the charts shows well definedboundaries for the Galilean-type on the lower endof its scaling for fixed sensor size due to the magnifi-cation constraint (Mopt ≥ 2). The curves also show theeffect of the sensor packing constraint (Eq. (19)) withthe dashed line representing an ideal sensor withoutadditional packaging (ρ � 1) and the solid line repre-senting the more realistic situation with a sensorpackage twice as large as the active pixel array(ρ � 2). As expected this constraint is more severefor the Galilean-type arrangement since it is moreclosely packed than the Keplerian arrangement. Thisconstraint occurs for the Galilean-type at loweraggregate resolutions, while at larger resolutionsfor the Keplerian-type. This is due to the fact thatincreasedmagnification, which is generally necessaryfor a larger aggregate resolution, expands theGalilean-type and compacts theKeplerian-type archi-tecture. This analysis suggests that although theGalilean-type architecture is attractive due to itssmaller overall size, it is muchmore constrained thanthe Keplerian-type arrangement and more likely tobe impractical when items such as the optomecha-nics, electronics, focus mechanisms and so on areconsidered.

Figure 7 shows volume as a function of objectiveF∕# and magnification for 2 gigapixel (a) and 50gigapixel (b) Keplerian-type designs, again withthe same fixed sensor (Np � 2800, p � 1.4 μm,ρ � 2). These plots indicate the system scaling pos-sible by varying optical power and the apparentbroader design space at 2 gigapixels resolution,due to the minimum magnification (Eq. (15)) andsensor packing (Eq. (19)) constraints.

B. Comparison to a Multicamera Array

The importance of the MMS architecture for wide-angle high-resolution imagers is best demonstratedcompared to a multicamera array, where cameraswith independent optics each sample a portion ofthe FOV. Using the same starting parameters de-scribed in Sec. 2, each camera of the multicameraarray will have a focal length described by

fMCA � XS

2 tan�α� �20�

and an aperture diameter given by

DMCA � 0.61 ·Npλtan�α� : (21)

Assuming that the cameras are distributed on acurved surface, the area and volume of the multica-mera array depend primarily on the spacing of thecamera elements. For lower overall resolution sys-tems with larger angular field per sensor, α, thesensor package will limit the spacing. As the over-all system resolution increases with fixed identi-cal array sensors, there will be a point, F∕#MCA� 1∕2ρ tan�α�, where the diameter of the optics be-comes larger than the sensor package and domi-nates the camera spacing. The scaling factor ρ isonce again included to represent the size of thepackage compared to the pixel array; with typical

Fig. 7. (Color online) Effect of objective F∕# and telescopemagnification on optical volume for Keplerian-type designs at(a) 2 gigapixels and (b) 50 gigapixels aggregate resolution. Identi-cal fixed sensor size and packaging used for both (Np � 2800,p � 1.4 μm, ρ � 2).

10 July 2012 / Vol. 51, No. 20 / APPLIED OPTICS 4697

values of about 2 for the smaller array height inCMOS image sensors. The 2D area that the multi-camera array occupies is then estimated to be

AreaMCA;optics lim �

�32NADMCA sin�FOVT∕2�

�2

π�22�

when the camera spacing is dominated by the opticsaperture, and

AreaMCA;sensor lim �

�32NA�ρXS� sin�FOVT∕2�

�2

π�23�

when the camera spacing is limited by the sensorpackage. The optics volume of the multicamera ar-ray is estimated to be

VolumeMCA;optics lim � πfMCA

�NADMCA

2

�2

(24)

when the camera spacing is dominated the by theoptics aperture, and

VolumeMCA;sensor lim � πfMCA

�NA�ρXS�

2

�2

(25)

when the camera spacing is dominated by the sen-sor package. Figure 8 shows a comparison betweenthe multicamera and Keplerian MMS imager archi-tectures using 8 megapixel square focal planes with2800 pixels in 1D and a pixel pitch of 1.4 um. Be-cause of the Rayleigh criterion based on the pixelpitch, both architectures are F∕2.1 at the imageplane. This comparison assumes that both the mul-ticamera array and MMS imager had optimal sen-sor area formats. If both systems use a conventional4∶3 aspect ratio focal plane, the array camera vo-lume (and area) could be reduced by approximately35% by matching the horizontal and vertical anglepitches to the focal plane’s asymmetric shape.

In Fig. 8 there is crossover point where the areaand volume of the MMS lens architecture become ad-vantageous compared to the multicamera array. Thispoint occurs shortly after the transition point for themulticamera array, where the diameter of the opticsbegin to dominate the size of the array needed to re-solve the IFOV on the chosen pixel pitch. At resolu-tions significantly larger than the crossover, thescaling advantage of the MMS lens architecture be-comes dramatic, especially in diameter and area ofthe array. For example, Fig. 9 shows a size compar-ison between the multicamera array and MMSsystem at 50 gigapixels. At 50 gigapixels the multi-camera array measures roughly 6.2 m in diametercompared to a diameter of 1.6 m for the MMS systemusing the “optimal magnification” of Fig. 6(b) and an

F∕#o of 9 (chosen using the F∕#o rule of thumb fromSec. 3.A). Both example systems are F∕2.1 at the im-age plane. The simple explanation for this dramaticdifference is that the largest optical element isshared between all array elements in the MMS lensarchitecture, creating a significant advantage whenthe required aperture is much larger than the focalplane. At smaller resolutions, such as the 2 gigapixelexample in this paper, the comparison is less dra-matic, but it is clear that the MMS lens structurehas important advantages for very large pixel countwide-field cameras.

4. Design Example: MOSAIC 2 Gigapixel Camera

In this section we present the detailed design ofa MMS imager currently being constructed forDARPA’s AWARE program. The MMS design re-solves more than 2 gigapixels at an IFOVof 41.5 μradover a symmetric FOVof 120°, using 226 14megapix-el CMOS image sensors. In Sec. 3.A we showed thatthe MMS lens architecture has significantly betterscaling behavior compared to a multicamera arrayfor wide FOV, small IFOV systems due to its sharedlargest aperture. At 2 gigapixels, this advantage isless clear with a first-order MMS having a smallerarea but larger volume than a first-order multica-mera array (Fig. 8). This design serves as a practical

Fig. 8. (Color online) Area and volume as a function of 2D resolu-tion comparing the Keplerian MMS imager architecture (bluecurves) and the multicamera array architecture (red curve) forfixed sensor size (Np � 2800, p � 1.4 μm, ρ � 2). (a) 2D projectedoptical area, and (b) optical volume. Both system architectures areF∕2.1 at the image plane.

4698 APPLIED OPTICS / Vol. 51, No. 20 / 10 July 2012

first demonstration of the concept and a step towardeven larger pixel count wide-FOV systems. We willconstrain this section to a discussion of the specificsof the optical design. A preliminary discussion of thedigital image reconstruction and array stitching forthis camera can be found elsewhere [24].

A. Design and Optical Performance

The monocentric triplet objective of this design con-sists of a fused silica (crown) ball lens core and anOhara SNBH8 (flint) external shell optimized forminimal spherical and axial chromatic aberrationusing ZEMAX [25]. The radii of curvature of the frontand back SNBH8 shells differ slightly—31.8 mm and33.378 mm respectively—giving a small front-to-backasymmetry, which provides an improved solutionfor the object at infinity. The objective lens has aneffective focal length of 70.25 mm, and focuses to a

spherical intermediate image plane 70.25 mm fromthe center of the objective as shown in Fig. 10(a).

The aspheric subimagers that form the array of theMMS lens are shown in Fig. 10(b). Each subimager inthe array is made from the same plastic lens partsreplicated by injection molding. The first elementof the subimager, which functions as the eyepiece,is a plastic cemented doublet of polycarbonate andZeonex E48R [26] with a hexagonal aperture for closepacking at the front plane as shown in Fig. 10(c).With the exception of the center pentagonal element,which must be custom shaped, all of the elements inthe array have hexagonal packing. The hexagonalshape of the front element improves the RI for fieldangles that fall at the vertex of a hexagonal packingscheme, where three subimagers share the circularcone of light from the objective.

The array camera is made of an air-spaced pair ofZeonex E48R and polycarbonate lenses to providethe final image. The front side of the Zeonex lens con-tains a weak diffractive optical element (DOE) to aidwith the chromatic correction of the subimager. Theoptimized DOE has 36 periods with a minimum dif-fractive groove pitch of 70 μm suitable for diamondturning and injection molding. The polycarbonatelens of the array camera is a thick meniscus element,functioning both to flatten the field and control thechief ray angle (CRA) to best match the microlensesof the Aptina MT9F001 1∕2.3 inch CMOS image sen-sor used with this design [27]. This image sensor has4608 × 3688 1.4 μm pixels, with gradually shifted mi-crolenses up to 25° CRA at the sensor’s maximumimage height.

The stop for this design is 6.4 mm in diameter, lo-cated at the front of the array camera to provide ac-ceptable RI (>68%) across the full stitched image asdescribed in Sec. 3. Figure 11 shows performance re-sults: incoherent polychromatic modulation transfer

Fig. 9. Size comparison at 50 gigapixels aggregate resolutionover 120° symmetric field of view. (Left) 50 gigapixel multicameraarray: 6,360 independent 8 megapixel cameras each 84 × 168 mm.6.2 m diameter for full 120° concave array. (Right) 50 Gigapixelmonocentric multiscale imager: 6,360 common aperture 8 mega-pixel imagers. 1.6 m system diameter at M � 5.8, F∕#o � 9. Bothsystem architectures are F∕2.1 at the image plane.

Fig. 10. (Color online) Layout of the MOSAIC 2 gigapixel MMSlens design. (a) Multiscale arrangement showing monocentricobjective and surrounding subimagers; (b) 4-element injection-molded subimager layout, and (c) perspective view showing hexa-gonal tiling and light sharing of subimagers.

Fig. 11. (Color online) Single channel performance for theMOSAIC 2 gigapixel MMS lens system. (a) Polychromatic MTF(0.47, 0.55, 0.65 μm); (b) Relative illumination over field requiredin vertex direction; (c) Through-focus polychromatic MTF at90 cyc∕mm (¼ Nyquist), and (d) through-focus MTF at 178 cyc∕mm(½ Nyquist). (a), (c) and (d) show Tangential and Sagittal data forfield angles of 0° (blue), 1° (green), 2° (red), 3° (light green), 4° (purple)and 4.6° (cyan).

10 July 2012 / Vol. 51, No. 20 / APPLIED OPTICS 4699

function (MTF), RI, and through-focus MTF for asingle channel of the MMS lens out to a field angleof 4.6° (9.2° FOV) in the vertex direction of the hex-agonal packing. This value of 4.6° was determined tobe the FOV required per subimager to provide sub-stantial overlap between adjacent subimagers in thearray based on the geodesic packing frequency, i.e.,the number of subimagers contained in the 120° fullsystem FOV. The minimum center-to-center angularseparation between subimagers with this geodesicpacking scheme was 7.04°. The magnification ofthe system was designed such that the required im-age formed by each subimager fell just inside an in-scribed hexagon within the active area of each imagesensor. The transverse magnification between the in-termediate image plane of the objective and the finalimage plane is −0.484 (demagnified). The workingF∕# for this design is 2.19 on-axis, increasing to 2.61at the 4.6° edge of the subimager’s field due to al-lowed vignetting. Distortion at the edge of the fieldis −3.3% (barrel) to be corrected in final post-processing.

B. Tolerances and Refocus

The monocentric objective is made using standardspherical lens polishing technology and can be formedwith either three elements, where the central fused si-lica ball lens is formed monolithically, or four parts,where two hemispherical halves of the ball lens arebonded together. The second of these two optionsallows for the possibility of an internal stop to be in-cluded in the fabrication of the objective. The toler-ances for the monocentric objective are �25 μmdecenter or departure from the monocentric condition,and ¼λ irregularity.

The injection-molded lenses of the subimager aremounted into lens barrels to fix the relative positionsof the parts. The design of the subimager was opti-mized for as-built performance with the following tol-erances suitable for injection molding: �12.5 μmsurface/element decenters, �12.5 μm glass/air axialthickness, �0.05° surface/element tilt, and �0.001material index of refraction [28]. Using these toler-ances, a root-sum-square sensitivitymethod estimatesa root-mean-squared (RMS) wavefront change fromthe nominal 0.095 to 0.139. Monte Carlo analysis ofthe system suggests reasonable insensitivity to theabove tolerances maintaining on-axis MTF > 0.65,70% field MTF > 0.38, and 90% field MTF > 0.15 athalf Nyquist (178 cyc∕mm). At full Nyquist, on-axisMTF > 0.28, 70% field MTF > 0.15, and 90% fieldMTF > 0.05 at 357 cyc∕mm.

The positioning tolerance of each subimager’s bar-rel in relation to the monocentric objective was foundto be �300 μm decenter (or monocentric axis devia-tion),�150 μm axial, and�0.12° tilt. Although thesevalues are large, they represent local tolerances forsubimager performance and do not ensure completeimage overlap for processing of the final image. Sys-tem (global) alignment tolerances are more complexsince they depend on the geodesic packing, sensor

size, and orientations and the tolerance stack upwithin each subimager. These issues will be dis-cussed further in a subsequent paper describingconstruction of the MOSAIC camera system.

Simple refocus is accomplished by axial position-ing of the CMOS image sensor at the back of eachsubimager’s lens barrel. This design is capable of re-focus over a range from infinity to 15 m with a�70 μm change in the axial position of the sensor.This range can be improved to better than 5 m ifthe focus adjustment is moved to adjust the spacingbetween the last two elements of the subimager asopposed to the sensor position.

C. Comparison to First-Order Model

Table 1 shows a parameter comparison between a 2gigapixel afocal first-order model, and a modifiedfirst-order model describing the MOSAIC design ofthis section. In this case, the MOSAIC design pre-ceded the first order model and was not constrainedto be afocal at the stop. Instead it is almost afocal,making a somewhat loose but useful comparison pos-sible. To compare as closely as possible, the firstseven parameters of Table 1 were matched as designstarting parameters using the flat-to-flat subimagerorientation of the final design. The parameters in theleft-hand column were calculated from the afocalparameters of Secs 2 and 3 while the first-order para-meters for the MOSAIC design in the middle columnwere calculated from the lens elements of the finaldesign. The first-order afocal model shows goodagreement with the final design with notable differ-ences occurring in the objective-eyepiece distance (d),and the focal length (f AC) of the array camera, whichdisplay the deviation from the afocal condition.These changes account well for the increased opticalvolume and vignetting [see Fig. 11(b)] of theMOSAIC design compared to the first-order model.

Table 1. Comparison Between First-Order Model Parameters for the 2Gigapixel MOSAIC Design

First-OrderAfocal Model

Modified First OrderModel (not afocal)

MOSAICDesign (actual)

FOVT 120° 120°IFOV 41.5 μrad 41.5 μradα 3.4° 3.4°p 1.4 μm 1.4 μmXS 4.0 mm 4.0 mmF∕#o 4.4 4.4M 2.5 2.5F∕#AC 2.1 2.1Do 33.2 mm 33.9 mmf o 73.4 mm 70.3 mmf EP 29.4 mm 27.7 mmDEP 12.5 mm 12.5 mmd 103 mm 111 mmlrelief 20.8 mm 19.5 mmf AC 13.3 mm 24.6 mmDAC 6.1 mm 7.0 mmMr

a−0.45 −0.47

Volume 4702 cm3 6915 cm3 7043 cm3

aMagnification between intermediate image and final image.

4700 APPLIED OPTICS / Vol. 51, No. 20 / 10 July 2012

Comparing the volume of the modified first-ordermodel to the actual volume of MOSAIC final design,shown in the right-hand column of the Table 1, yieldsan error of less than 2%. The complete optical pre-scription data for the 2 gigapixel MOSAIC camerais included in Table 2, Table 3, and Table 4 below.

5. Conclusion

In this paper, we have developed a systematic ap-proach to the design of monocentric multiscale ima-gers including first-order geometrical optical effects,especially vignetting, to specify candidate systemsfor full optical design. We have identified two classesof MMS imagers, those with and without and inter-nal image plane, and shown how the optical systemvolume scales with the key parameters of component

lens F∕# and telescopic magnification. We have illu-strated this design space with a fully tolerancedoptical design of a diffraction limited MMS lens,which is the basis of a 120° FOV, 41.5 μrad IFOV,2 gigapixel imager currently under construction.We conclude that with further development of com-putational image processing, and with the use of ad-vanced fabrication and assembly techniques, MMSlens architectures show promise for wide-angleimagers with aggregate resolutions to well over 50gigapixels.

This work has been supported by DARPA via theAdvanced Wide FOV Architectures for Image Recon-struction (AWARE) program, and is part of a collabora-tive effort on real-time gigapixel imagers includingresearchers from Duke, UCSD, RPC Photonics, Inc.,

Table 2. Single Path Prescription Data for 2 Gigapixel MOSAIC Camera (Sec. 4)

Surf. # Comment Rad of Curv. (mm) Thick. (mm) Material Diam. (mm)

1 Ball lens 31.8 13.613 S-NBH8 63.62 18.187 18.187 F_Silica 36.3743 Infinity 18.187 F_Silica 36.3744 −18.187 15.191 S-NBH8 36.3745 −33.378 36.878 Air 66.7566 Int. image Infinity 33.226 Air —

7 Eyepiece 22.51297 8.386424 Polycarb 14.4a

8 8.077042 8 E48R 14.4a

9 −25.16423 18.527 Air 14.4a

10 IR cut Infinity 1 Bk7 811 Stop Infinity 0 Air 6.412b Array camera 8.097214 9.284709 E48R 713 −43.31043 2.482625 Air 6.614 −66.75502 3.374655 Polycarb 5.615 4.710791 1 Air 5.33216 Focus Infinity 0.00816865 Air 5.13917 Sensor Infinity 0.4 Bk7 718 Infinity 0.125 Air 720 Image Infinity — Si 6.6aVertex-to-vertex distance on hexagonal aperture.bSee Table 4 for DOE prescription.

Table 3. Single Path Prescription Data for 2 Gigapixel MOSAIC Camera (Sec. 4): Aspheric Terms

Surf. # Comment r4 Aspheric Term r6 Aspheric Term r8 Aspheric Term r10 Aspheric Term

7 Eyepiece −2.8147152 × 10−5 −4.0237714 × 10−7 4.4135689 × 10−9 —

8 7.2425919 × 10−5 −1.352871 × 10−5 1.0086073 × 10−7 —

9 −1.6062121 × 10−5 1.0507867 × 10−6 −1.2540803 × 10−8 —

12a Array camera −8.1858462 × 10−6 4.9291849 × 10−7 −6.1449905 × 10−8 5.1302673 × 10−9

13 0.00049936384 −6.6577601 × 10−6 6.2804293 × 10−7 3.7586356 × 10−8

14 −0.0026541057 0.00016173157 −5.9078789 × 10−6 4.0337124 × 10−7

15 −0.0065076023 0.00045055736 −4.2893373 × 10−5 3.5191145 × 10−6

aSee Table 4 for DOE prescription.

Table 4. Single Path Prescription Data for 2 Gigapixel MOSAIC Camera (Sec. 4): DOEa

Surf.# Comment Max Rad. Ap. (mm) p2“Binary 2” Term p4

“Binary 2” Term p6“Binary 2” Term

12 Array camera 4.4460065 −250.9302 18.164982 5.2014151aSee ZEMAX Optical Design Program User’s Manual, Chap. 11, p. 284 (Radiant ZEMAX LLC, July 8 2011).

10 July 2012 / Vol. 51, No. 20 / APPLIED OPTICS 4701

Distant Focus Corporation, the University of Arizona,MIT, Teledyne, Raytheon, SAIC, and Aptina. Thisoptical design work has been assisted by MichaelGehm, Dathon Golish, and Esteban Vera at UAZ;Jungsang Kim, Hui Son, and Joonku Hahn at Duke;Ron Stack and Adam Johnson at DFC; Bob Gibbonsat Raytheon; and Paul McLaughlin and Jeffrey Shawat RPC Photonics. The authors also acknowledge KateBaker for assistance with solid modeling. Finally theauthors acknowledge helpful discussions on image sys-tem architectures and scaling with Joseph Mait andPredragMilojkovic at the U.S. Army Research Labora-tory, and Ravi Athale at MITRE Corporation.

References and Notes1. R. Sargent, C. Bartley, P. Dille, J. Keller, I. Nourbakhsh, and R.

LeGrand, “Timelapse GigaPan: Capturing, sharing, andexploring timelapse gigapixel imagery,” presented at the FineInternational Conference on Gigapixel Imaging for Science,Pittsburgh, PA (2010).

2. V. Suntharalingam, R. Berger, S. Clark, J. Knecht, A. Messier,K. Newcomb, D. Rathman, R. Slattery, A. Soares, C. Stevenson,K. Warner, D. Young, L. P. Ang, B. Mansoorian, and D. Shaver,”A4-side tileable back-illuminated 3D-integrated Mpixel CMOSimage sensor, “in Proceedings of IEEE International Solid StateCircuits Conference Technical Digest (IEEE, 2009).

3. K. W. Hodapp, N. Kaiser, H. Aussel, W. Burgett, K. C. Cham-bers, M. Chun, T. Dombeck, A. Douglas, D. Hafner, J. Heasley,J. Hoblitt, C. Hude, S. Isani, R. Jedicke, D. Jewitt, U. Laux, G.A. Luppino, R. Lupton, M. Maberry, E. Magnier, E. Mannery,D. Monet, J. Morgan, P. Onaka, P. Price, A. Ryan, W. Sieg-mund, I. Szapudi, J. Tonry, R. Wainscoat, and M. Waterson,“Design of the Pan-STARRS telescopes,” Astron. Nachr.325, 636–642 (2004).

4. B. Leininger, J. Edwards, J. Antoniades, D. Chester, D. Haas,E. Liu, M. Stevens, C. Gershfield, M. Braun, J. D. Targove, S.Wein, P. Brewer, D. G. Madden, and K. H. Shafique, “Auton-omous real-time ground ubiquitous surveillance-imaging sys-tem (ARGUS-IS),” Proc. SPIE 6981, 69810H (2008).

5. A. W. Lohmann, “Scaling laws for lens systems,”Appl. Opt. 28,4996–4998 (1989).

6. B. Wilburn, N. Joshi, V. Vaish, E. Talvala, E. Antunez, A. Barth,A. Adams, M. Horowitz, and M. Levoy, “High performanceimaging using large camera arrays,” ACM Trans. Graph. 24,763–776 (2005).

7. D. J. Brady and N. Hagen, “Multiscale lens design,” Opt.Express 17, 10659–10674 (2009).

8. D.L.MarksandD.J.Brady,“Close-upimagingusingmicrocameraarrays for focal plane synthesis,”Opt. Eng. 50, 033205 (2011).

9. J. E. Ford and E. Tremblay, “Extreme form factor imagers,” inImagingSystems (Optical Society of America, 2010), paper IMC2.

10. D. L. Marks, E. J. Tremblay, J. E. Ford, and D. J. Brady, “Mi-crocamera aperture scale in monocentric gigapixel cameras,”Appl. Opt. 50, 5824–5833 (2011).

11. Michael Kidger, Fundamental Optical Design (SPIE, 2002),Chap. 7.

12. T. Sutton, “On the panoramic lens,” a paper read toBritish Association Report in 1861, as described in Patentsfor Inventions, Abridgements of the Specifications Relatingto Photography (Great Britain Patent Office, 1861).

13. D. L. Marks and D. J. Brady, “Gigagon: A monocentric lensdesign imaging 40 gigapixels,” in Imaging Systems (OpticalSociety of America, 2010), paper ITuC2.

14. G. Krishnan and S. K. Nayar, “Towards a true sphericalcamera,” Proc. SPIE 7240, 724002 (2009).

15. O. Cossairt, D. Miau, and S. K. Nayar, “Gigapixel computa-tional imaging,” in IEEE International Conference on Compu-tational Photography (IEEE, 2011).

16. H. Son, D. L. Marks, E. J. Tremblay, J. Ford, J. Hahn, R. Stack,A. Johnson, P. McLaughlin, J. Shaw, J. Kim, and D. J. Brady,“A multiscale, wide field, gigapixel camera,” in Imaging Sys-tems Applications (Optical Society of America, 2011), paperJTuE2.

17. George Smith and David A. Atchison, The Eye and VisualOptical Instruments (Cambridge University, 1997), Chap. 17.

18. W. C. Sweatt, D. D. Gill, D. P. Adams, M. J. Vasile, and A. A.Claudet, “Diamond milling of micro-optics,” IEEE Aerosp.Electron. Syst. Mag. 23, 13–17 (2008).

19. B. W. Clare and D. L. Kepert, “The closest packing of equalcircles on a sphere,” Proc. R. Soc. A 405, 329–344 (1986).

20. H. Kenner, Geodesic Math and How to Use It, 2nd ed.(University of California, 2003).

21. H. S. Son, D. L. Marks, J. Hahn, J. Kim, and D. J. Brady,“Design of a spherical focal surface using close packed relayoptics,” Opt. Express 19, 16132–16138 (2011).

22. O. S. Cossairt, D. Miau, and S. K. Nayar, “Scaling law for com-putational imaging using spherical optics,” J. Opt. Soc. Am. A28, 2540–2553 (2011).

23. http://mathworld.wolfram.com/SphericalCap.html.24. D. Golish, E. Vera, K. Kelly, Q. Gong, D. J. Brady, and M. E.

Gehm, “Image formation in multiscaleoptical systems,” inComputational Optical Sensing and Imaging (Optical Societyof America, 2011), paper JTuE4.

25. ZEMAX, Radiant ZEMAX LLC, 112th Avenue NE, Bellevue,Wash. 98004.

26. L. P. Zeon Chemicals, Louisville, Ky. 40211, http://www.zeonchemicals.com/.

27. Aptina Imaging, San Jose, Calif. 95134, http://www.aptina.com/

28. Michael Pfeffer, “Optomechanics of plastic optical compo-nents,” in Handbook of Plastic Optics, S. Bäumer, ed.(Wiley-VCH, 2005).

4702 APPLIED OPTICS / Vol. 51, No. 20 / 10 July 2012