Super-resolution for flash ladar imagery

Super-resolution for flash ladar imagery

Shuowen Hu,1,* S. Susan Young,1 Tsai Hong,2 Joseph P. Reynolds,3 Keith Krapels,3

Brian Miller,3 Jim Thomas,3 and Oanh Nguyen3

1Army Research Laboratory, 2800 Powder Mill Road, Adelphi, Maryland 20783, USA2National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, Maryland 20899, USA3Night Vision and Electronic Sensors Directorate, 10221 Burbeck Road, Fort Belvoir, Virginia 22060, USA

*Corresponding author: [email protected]

Received 20 August 2009; revised 16 December 2009; accepted 21 December 2009;posted 5 January 2010 (Doc. ID 115804); published 1 February 2010

Flash ladar systems are compact devices with high frame rates that hold promise for robotics applica-tions, but these devices suffer from poor spatial resolution. This work develops a wavelet preprocessingstage to enhance registration of multiple frames and applies super-resolution to improve the resolution offlash ladar range imagery. The triangle orientation discrimination methodology was used for a subjectiveevaluation of the effectiveness of super-resolution for flash ladar. Results show statistically significantincreases in the probability of target discrimination at all target ranges, as well as a reduction in subjectresponse times for super-resolved imagery. © 2010 Optical Society of America

OCIS codes: 100.6640, 110.3000.

1. Introduction

Ladar devices are a crucial component in robot appli-cations, providing real-time range data for obstacleavoidance and path guidance. Current small robotsgenerally employ a two-dimensional (2D) scanningladar that scans along a single line. In indoor urbanenvironments where the setting is highly clutteredwith overhanging objects, such as tabletops, the 2Dscanning ladar systems may not be sufficient fornavigation and obstacle avoidance because objectsabove or below the devices’ scanning line would notbe detected [1,2]. These devices are also bulky andheavy—the SICK LMS-200 device that has becomethe standard sensor for small robot research weighs4:5kg. A new generation of three-dimensional (3D)ladar devices, called flash ladar, offers a promisingsolution to small robot navigation in urban environ-ments, where modern warfare is often conducted.Flash ladar devices are compact and lightweight sen-sors that acquire a 3D range image of the surroundingenvironment. The SwissRanger SR-3000 (CSEM,

Switzerland) flash ladar device (Fig. 1) used in thisstudy weighs only 162 g. It emits diffuse modulatednear-infrared light to measure the subsequent phaseshift between the original emitted light and the re-flected light in order to calculate range based onthe time-of-flight principle [3]. The detector utilizedby flash ladar devices is a focal plane array (FPA),which is typically limited to a maximum size of 256 ×256 detectors. Consequently, these devices cannotachieve the resolution of scanning ladar systems.To improve the spatial resolution of flash ladar sys-tems, super-resolution can be applied to complementthe compact and 3D imagery advantages of flashladar devices. The objectives of this work are (a) toprovide an objective assessment of super-resolutionimprovement for flash ladar imagery using sensorspecifications and spectral analysis of original versussuper-resolved images, (b) to develop a preprocessingstage for flash ladar imagery to improve frameregistration for super-resolution, (c) to apply super-resolution for flash ladar imagery, and (d) to obtaina measure of the quality improvement achieved withsuper-resolution using the triangle orientation discri-mination (TOD) methodology. Two types of measuresare used for quality assessment: objective measures

0003-6935/10/050772-09$15.00/0© 2010 Optical Society of America

772 APPLIED OPTICS / Vol. 49, No. 5 / 10 February 2010

typically involvenumerical calculations of a quantita-tive metric, while subjective measures are acquiredthrough human perception experiments and thusare expected to correlate well to human-based perfor-mance. The TODmethodology employed in this workprovides a subjective measure of the performance im-provement achieved with super-resolution of flashladar imagery.Super-resolution algorithms utilize a series of low-

resolution frames to generate a higher-resolution im-age, and are typically composed of two major stages:the registration stage and the reconstruction stage.During the registration stage, the shift of a givenframe with respect to a reference frame (usuallythe first frame of the series) is computed to subpixel(i.e., decimal pixel) accuracy, and the second stageutilizes this subpixel information to interpolate thelow-resolution frames onto a higher-resolution grid.A necessary condition for successful super-resolutionis the presence of differing noninteger shifts betweenframes to provide distinct information from whichthe super-resolved imagery can be reconstructed.Previous work by [4] applied the super-resolutionalgorithm of [5] to flash ladar data, and observed im-provement in image quality in terms of number ofedges detected. In this work, the super-resolution al-gorithm of [6] is applied to flash ladar imagery. Thisalgorithm separates the registration stage into agross shift (i.e., integer pixel shift) estimation stageand a subpixel shift (i.e., decimal pixel shift) estima-tion stage for improved registration accuracy [6].Both substages use the correlation method in the fre-quency domain to estimate shifts between the frameseries and the reference image. The reconstructionstage of [6] applies the error-energy reduction meth-od with constraints in both spatial and frequencydomains to generate a high-resolution image.Since flash ladar imagery is inherently smoother

than visible-light imagery (flash ladar data does

not capture the texture or color of the scene that givesvisible-light imagery much of its high frequency com-ponents), this work develops a preprocessing stagespecific to flash ladar for improved image registra-tion. Awavelet edge filtering method [7] and a Cannyedge detectionmethod [4] were investigated and com-pared to the accuracy achievedwith no preprocessing.Results show that the wavelet preprocessing methodachieved the best shift estimation accuracy for flashladar data and, hence, it was used to preprocess theacquired range imagery. Following preprocessing,super-resolutionwasapplied to increase the samplingrate of the flash ladar sensor’s focal plane array, effec-tively increasing the sampling rate of the range datain the x and y directions (where range is in the z direc-tion). Therefore, by utilizing a series of range imageryframes that contain subpixel shifts (generated bynatural jitter when the sensor is mounted on a mov-ing platform or held by a person), super-resolutioncan reconstruct high-resolution range imagery. Toassess the improvement in flash ladar imagery ac-hieved with super-resolution, the authors used theTOD methodology [8,9] to obtain a human sub-jective measurement of quality. The TOD task is afour-alternative forced-choice perception experimentwhere the subject is asked to identify the orientationof a triangle (apex up, down, right, or left) [9], and canbe used to assess the probability of target discrimina-tion, as well as response times. Results show that theprobability of target discrimination and responsetimes improved significantly at almost all targetranges with super-resolved flash ladar imagery.

2. Methodology

A. SwissRanger SR-3000 Flash Ladar Camera

The SwissRanger SR-3000 flash ladar camera(CSEM, Switzerland) was utilized for this work. Thesensor specifications are:

f -number ¼ 1:4,aperture diameter ¼ 2:24 × f -number

× wavelength ¼ 2:67 μm,FOV ¼ 47:5° × 39:6°,detector size ¼ 40 μm, andarray ¼ 176 × 144.

The camera uses a bank of 55 diodes to emit diffuse850nm near-infrared light modulated at a frequencyof 20MHz for a resulting nonambiguity range of7:5m. The camera can capture images at a maximumrate of 50 frames per second, variable with respect tothe user-set integration time. The SR-3000 providesboth range and intensity imagery, of which only theacquired range data was used for this study.

B. SwissRanger SR-3000 Characteristics Benefitting fromSuper-Resolution

FromthesensorspecificationslistedinSubsection2.A,the SR-3000 flash ladar camera is seen to be under-sampled, as the sample size (which is similar to the

Fig. 1. (Color online) SwissRanger SR-3000 flash ladar camera(Mesa Imaging, Switzerland).

10 February 2010 / Vol. 49, No. 5 / APPLIED OPTICS 773

detector size) is larger than the optics diffraction spot(which is similar to the aperture diameter). Over-sampled imaging devices do not benefit from super-resolution, but undersampled devices increasinglybenefit from super-resolution as the degree of under-sampling increases. Based on the analysis of charac-teristics of infrared imaging systems that benefitfrom super-resolution [4], flash ladar devices can begeneralized into two different regions: an under-sampled region where super-resolution is beneficial(where the detector size is bigger than the aperturediameter), and an oversampled region where super-resolution provides no benefits (where the detectorsize is smaller than the aperture diameter), as illu-strated in Fig. 2. The SR-3000 flash ladar device, withf -number of 1.4 and sample spacing of40 μmis locatedinthetopleft cornerofFig.2,wheresuper-resolution isexpected to provide significant benefits.To furtherprovideanobjectiveassessment of super-

resolution improvement for flash ladar imagery, spec-tral analysis was conducted using low-resolutionoriginal flash ladar imagery and super-resolved ima-gery. Let Eqs. (1) and (2) define the cumulative powerspectrum in wavenumber kx and ky, respectively:

S1ðkxÞ ¼Xky

jFðkx; kyÞj2; ð1Þ

S2ðkyÞ ¼Xkx

jFðkx; kyÞj2; ð2Þ

whereFðkx; kyÞ is the Fourier transform of the consid-ered image.Figure 3(a) the original, Fig. 3(b) shows the super-

resolved flash ladar imagery, and Fig. 4 displays theoverlaid ky-domain spectrums of the original and

super-resolved imagery in decibel scale. As can be ob-served from Fig. 4, the super-resolved image recoversthe high-frequency band from the aliased originalimage. This bandwidth recovery represents the reso-lution improvement achieved with super-resolutionof the SR-3000 flash ladar camera.

C. Preprocessing Stage for Improved Frame Registration

The purpose of the preprocessing stage is to empha-size edges in flash ladar imagery for improved frameregistration. One investigated method was the use ofmultiscale edge-wavelet transform [10] with themother wavelet, defined as an edge detector to calcu-late the horizontal and vertical partial derivatives ofthe input frame series at the second wavelet scale foreach frame. The partial derivatives of each framewere subsequently combined using sum of squaresto produce a wavelet edge-enhanced frame series.The other investigated preprocessing method wasthe use of Canny edge detection to generate a binaryedge frame series. The Matlab function edge(‘inputimage’, ‘canny’) was used to implement Canny edge

Fig. 2. (Color online) Near-infrared regions of super-resolutionbenefit, with SR-3000 flash ladar located in the top left region,where super-resolution provides significant benefit.

Fig. 3. Gray-scale range imagery of (a) original TOD target imageat range of 5m, and (b) super-resolved image.

Fig. 4. The ky-domain spectrum of the original flash ladar imageoverlaid with the spectrum of the super-resolved image displayedin decibel scale, showing that the super-resolved image recoversthe high-frequency band from the aliased original image.


detection, with parameters “percent of pixels notedges” set at 0.7 and “threshold ratio” set at 0.4.The following procedure was used to conduct syn-

thetic experiments in assessing which technique(wavelet or Canny) was the most effective prepro-cessing method for flash ladar imagery. First, a simu-lated high-resolution reference image was generatedby upsampling an oversampled (i.e., nonaliased)scanning ladar reference image (204 × 204 pixels)by a factor of 8 (to 1632 × 1632 pixels) using a Fourierwindowing method [10]. The high-resolution refer-ence image was then subsampled every m pixels inboth dimensions of the high-resolution reference im-age to generate a synthetic low-resolution image si-mulating low-resolution flash ladar imagery. A rangeof m (4, 8, 12, 16, 20, 28, 36, 48, 56) was utilized toproduce undersampling factors of m=8 (0.5, 1, 1.5, 2,2.5, 3.5, 4.5, 6, 7, respectively), simulating differentdegrees of aliasing. To create a synthetic frame serieswith known subpixel shifts for a givenm, the startingpixel position of subsampling on the high-resolutionreference image was varied according to a uniformrandom distribution and repeated to generate 30frames. The result was a set of frame series withknown subpixel shifts and different degrees of alias-ing. Figure 5 illustrates aliasing by first showing theunaliased spectrum of a discrete signal (for example,a scanning ladar image) produced by oversampling acontinuous space signal at a sampling frequencygreater than the Nyquist frequency (top plot). Ifthe sampling frequency is below Nyquist (simulatedby subsampling the reference image), the spectrumof the sampled signal is aliased with distorted high-er-frequency components as depicted in the bottomplot of Fig. 5.After generating the set of undersampled synthetic

frame series, preprocessing using either the waveletor the Canny method was applied. Subpixel shifts es-timated from the preprocessed frame series and theoriginal frame series were compared to the knownshifts to assess the effectiveness of preprocessing ver-sus no preprocessing. Let εi ¼ ðεxi; εyiÞ denote the re-gistration error vector of the ith frame, where εxi and

εyi are the registration errors in the x and y direc-tions; then the mean absolute error (MAE) was cal-culated for each synthetic frame series using Eq. (3),where n ¼ 30 is the number of frames in each frameseries:

E ¼ 1n

Xn

i¼1

‖εi‖ ¼ 1n

Xn

i¼1

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiε2xi þ ε2yi

q: ð3Þ

The calculated registration errors for each preproces-sing method were then used to assess which techni-que is the most effective for flash ladar imagery.

D. Triangle Orientation Discrimination Methodology

To assess the improvement achieved with super-resolution of flash ladar imagery in conjunction withpreprocessing, the TODmethodology [9] was applied.The TOD methodology, developed by NetherlandsTNO-FEL Laboratory, is a perception study that al-lows human subjects to provide a measure of imagequality at various target ranges. The test pattern isan equilateral triangle in one of four possible orien-tations (apex up, down, left, or right), and the mea-surement process is a four-alternative forced-choicepsychological procedure that requires the observerto indicate the orientation. Variation of triangle con-trast/size by changing target ranges results in a cor-rect discrimination percentage between 25% (pureguess) and 100%. Probabilities of target discrimina-tion can be calculated to provide a human subjectivemeasure of quality for both the original and super-resolved imagery at different target ranges.

The TOD methodology is suitable for electro-optical and optical imaging systems, and has beenwidely used in thermal and visual domain imagers.This methodology provides a simple task that has aclose relationship to real target acquisition with re-sults that are free from observer bias [8,9]. The TODmethodology was adapted to flash ladar data byusing a target consisting of a square white foamboard target (50 cm × 50 cm) with an equilateral tri-angular hole (7:5 cm per side) cut into the center ofthe board, as shown in Fig. 6.

E. Data Collection

Data collection for the experiment was conducted ata laboratory in the National Institute of Standardsand Technology. The SwissRanger SR-3000 flash la-dar camera was placed 6:5m from a beige wall, asdepicted in Fig. 6, and the target was positioned 3,3.5, 4, 4.5, 5, 5.5, and 6m in front of the camera.The investigated ranges were limited to between 3and 6m because flash ladar cameras exhibit inaccu-rate behavior at very close and very far target dis-tances relative to the operating range of thecamera [11]. At each target range, the triangle waspositioned in one of four possible orientations (apexup, down, left, or right) with the center approxi-mately 1m high. For each orientation at each range,four trials were conducted, each consisting of a se-quence of 32 frames acquired by holding the camera.

Fig. 5. (Color online) (Top) Unaliased spectrum of signal sampledabove Nyquist frequency, (mid) at Nyquist frequency, and (bot-tom) aliased when sampled below Nyquist frequency.


The natural motion of the hand while holding thecamera provided the subpixel shifts necessary forsuper-resolution and was assumed to be limited totranslations in the x (horizontal) and y (vertical)planes. Though slight rotation and translation in thez plane (distance) might have occurred from holdingthe camera, these parameters were not considered inthe current study.

F. Stimulus Strength

The variation in target range results in a variation ofstimulus strength. The relationship between stimu-lus strength and correct score is referred to as a psy-chometric function, which is measured empiricallyby the TOD methodology. Both the target contrastand the target size affect stimulus strength. In thisexperiment, stimulus strength is increased by in-creasing both the target size and the target contrast.Let the range contrast of the target be defined by thefollowing equation, where RH is the range of the tri-angular hole (always 6:5m in this experimental set-up) and RB is the range of the square foam board:

CR ¼ RH − RB

RH þ RB· 100: ð4Þ

For RB ¼ f3; 3:5; 4; 4:5; 5; 5:5; 6gm, the correspondingrange contrasts are CR ¼ f36:8; 30; 23:8; 18:2; 13;8:3; 4g. Therefore, as the target distance decreases,both the target contrast and the target size increase,for an overall increase in stimulus strength.

G. Integration Time

Integration time is a key setting for the SR-3000flash ladar camera. The integration time has to beset sufficiently high so that the detectors receive en-ough reflected light from the target to make an accu-rate calculation of range, but yet not so high as toproduce specular effects (i.e., saturation of the detec-tors). [12] found that the optimal integration time foreach target range lies within a very flat bowl-shapedcost function with sharp rises at the saturation andinsufficient light regions. Therefore, as long as the

integration time is set to receive enough reflectedlight without saturating the detectors, the settingwill be close to optimal. For the investigated targetranges RB ¼ f3; 3:5; 4; 4:5; 5; 5:5; 6gm, the corre-sponding integration time settings were T ¼ f15; 15;15; 20; 20; 20; 25gms—no saturation effects were ob-served at these integration times for all investigatedtarget ranges.

H. Super-Resolution of Flash Ladar Imagery

For each acquired frame series, the super-resolutionalgorithm of [6] was applied. First, the registrationstage of the super-resolution algorithm utilized thepreprocessed frame series, with the first frame ser-ving as the reference frame from which pixel shiftswere calculated for successive frames. Note that onlythe registration stage of the super-resolution algo-rithm utilized the preprocessed flash ladar data inorder to generate improved pixel shift estimates;the reconstruction stage of the super-resolution algo-rithm then used these shift estimates and theoriginal (i.e., unpreprocessed) flash ladar data to re-construct high-resolution flash ladar imagery usingthe error-energy reduction method [6]. The stoppingcriterion for the iterative error-energy reduction pro-cedure was defined to be when the decibel ratio of theimage energy of the current iteration and the errorenergy between the current and previous iterationdiffer less than a given threshold, set to be 0.0001for this work. Note that, although 32 frames were ac-quired for each frame series, only the first 25 frameswere utilized for super-resolution because the qualityof the super-resolved image did not improve once thenumber of processed frames exceeded 25. This is con-sistent with the work of [6], which showed that thequality of the super-resolved imagery ceased to im-prove once the number of frames exceeds a certainvalue, because the bandwidth recovery by super-resolution is limited by the bandwidth of the sensoror the target signature spectrum. The use of 25frames consequently resulted in a resolution im-provement factor of 5 in each direction for the super-resolved flash ladar imagery. To ensure that themonitor modulation transfer function (MTF) wasnot a limiting factor in the experiment, the super-resolved images (250 × 250 pixels) were bilinearly in-terpolated by a factor of 2 to 500 × 500 pixels, and theoriginal imagery (50 × 50 pixels) was also bilinearlyinterpolated to 500 × 500 for consistency betweenbaseline and super-resolved imagery.

I. Perception Experiment

The perception experiment was a four-alternativeforced-choice procedure (up, down, left, right). Thegray-scale baseline flash ladar range imagery wasgrouped into seven cells corresponding to the sevendifferent target ranges. Each cell had 16 originallow-resolution flash ladar images ð4 orientations ×4 trialsÞ. Similarly, the gray-scale super-resolvedrange imagery was grouped into seven cells with16 images each. The experiment therefore consisted

Fig. 6. (Color online) Triangle orientation discrimination targetconsisting of an equilateral triangle (7:5 cm per side) cut into asquare board (50 cm × 50 cm).


of 14 cells with a total of 224 images (layout and nam-ing convention shown in Table 1).Ten subjects (eight soldiers and two civilians)

participated in the experiment in August 2008 atthe perception laboratory in the U.S. Army’s NightVision and Electronic Sensors Directorate, withthe approval of the Human Subjects ResearchReview Board at the U.S. Army Medical Researchand Materiel Command. The subjects were shownone image at a time, with randomized presentationof cells and randomized presentation of images with-in each cell to reduce subject bias. The display moni-tors (Samsung SyncMaster 204B) had a resolution of1600 × 1200 pixels with a pixel pitch of 0:255mm×0:255mm.

3. Results and Discussion

A. Assessment of Registration Accuracy

Figure 7 shows the MAE of registration at each un-dersampling factor for the synthetic experiments,with the unit of error defined as a fraction of a pixel.Wavelet preprocessing was especially effective at lowand moderate degrees of aliasing (undersamplingfactor < 3:5), and continued to outperform both theCanny method and no preprocessing method untilan undersampling factor of 6. For severe aliasing (un-dersampling factor > 6), no preprocessing (i.e., usingoriginal imagery) resulted in the highest registrationaccuracy. This observed trend is expected; because la-dar data is characteristically smooth due to the lackof texture information, edge enhancement with thewavelet method will improve registration, but ifthe data is so severely undersampled that its high-er-frequency components are corrupted by aliasing,

then wavelet edge filtering (which uses theseseverely corrupted frequency components) will resultin poorer registration. The degree of aliasing in therange imagery acquired with the SwissRangerSR-3000 is expected to be in the moderate range,as results show that super-resolved images usingwavelet preprocessing yielded fewer artifacts thanthose without preprocessing.

B. Super-Resolved Versus Original Imagery

Figure 8 shows gray-scale and color-coded range ima-gery of the TOD target oriented “up” at a distance of5m from the camera. The orientation of the equilat-eral triangular hole is difficult to discern in theoriginal images at this distance, as the triangularhole appears like a blurred circle. By contrast, theorientation is clear in the super-resolved imagery.For imagery with target distances greater than5m, the orientation was yet more difficult to discernusing the original flash ladar imagery, but super-resolution at these greater distances continued tobe effective. Figure 9 shows gray-scale and color-coded range imagery of the TOD target oriented “left”at a distance of 4m from the camera. As the targetdistance decreases, the orientation of the triangularhole became more visible in the original imagery, butthe triangular hole still appeared distorted comparedto that of the super-resolved imagery.

Table 1. Cell Format and Naming Convention for Each Target Range

Range (m) A (3) B (3.5) C (4) D (4.5) E (5) F (5.5) G (6)

Original images AA BA CA DA EA FA GASuper-resolved images AB BB CB DB EB FB GB

Fig. 7. (Color online) Registration accuracy in terms of MAEobtained with wavelet preprocessing, Canny preprocessing, andno preprocessing at different undersampling factors.

Fig. 8. (Color online) (Top) Gray-scale range imagery and (bot-tom) color-coded range imagery for (left) original image and(right) super-resolved image of TOD target at range of 5m.


C. Probability of Target Discriminationand Response Times

The chance-corrected group-averaged probability oftarget discrimination at each target range is shownin Fig. 10 for super-resolved and original flash ladarimagery. Equation (5) was used to correct for theguess rate of 25% (Pg ¼ 0:25):

Pcorrected ¼ P − Pg

1 − Pg: ð5Þ

At all target ranges, super-resolution imagery pro-duced a higher probability of target discriminationwith smaller intersubject variability. At a targetrange of 3m, the original imagery resulted in a73% of the probability of target discrimination, while

the super-resolved imagery reached 100%—targetdiscrimination performance improved by 37% withsuper-resolution. As target distance increased, sub-jects had more difficulty in discriminating the targetorientation using the original imagery. At a targetdistance of 6m, the original imagery resulted in aprobability of target discrimination of 25%, whilethe super-resolved imagery reached 95%, resultingin a 280% improvement in target discrimination per-formance with super-resolution. Therefore, from 3 to6m, target discrimination performance exhibited animprovement of 37% to 280%, respectively, by apply-ing super-resolution.

Not only were subjects able to achieve higher accu-racy at all target ranges with super-resolved ima-gery, but response times were also faster with lessvariability for super-resolved imagery at all rangesunder 6m. Figure 11 shows the group-averaged re-sponse times at each range with standard error barsrepresenting intersubject variability. At a range of5m, subjects responded in an average time of 1:39 susing the super-resolved imagery, 65% faster thanthe response time using original imagery. Super-resolution resulted in a minimum speedup of 48%(at 3m) for all target ranges, with the only exceptionof 6m. At 6m, response times for original imagerywere almost the same as the response time forsuper-resolved imagery, possibly because the taskof identifying orientation with the original imagerybecame so difficult at 6m that the subjects decidedto quickly pick a random orientation for the originalimagery.

D. Statistical Assessment of Improvement

To assess whether super-resolution resulted in astatistically significant improvement over the origi-nal imagery, dependent samples (i.e., paired) t-testwas applied for each target range. The paired t-test,implemented using Eq. (6), is commonly used in sta-tistics to test the null hypothesis that the differencebetween two conditions (in this case, the probabilityof target discrimination of super-resolved versus ori-ginal imagery) has a mean value of zero. In Eq. (6),

Fig. 9. (Color online) (Top) Gray-scale range imagery and (bot-tom) color-coded range imagery for (left) original image and(right) super-resolved image of TOD target at range of 4m.

Fig. 10. (Color online) Chance-corrected probability of targetdiscrimination at each range with standard error bars showing in-tersubject variability.

Fig. 11. (Color online) Average subject response times withstandard error bars showing intersubject variability.


the subscripts A and B denote the two differentgroups to be compared (original and super-resolvedimagery), x represent the data sample (empiricalprobability of target discrimination), N ¼ 10 is thenumber of subjects, and t is the computed t statistic[13]:

t ¼ �xA − �xBffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiVarðxAÞþVarðxBÞ−2CovðxA;xBÞ

N

q : ð6Þ

Associated with the t-statistic is a p-value[p ¼ PðT ≥ tÞ, where T is the Student’s t-distribution]representing the probability of obtaining a result atleast as extreme as the observed result under theassumption that the null hypothesis is true. The p-value can then be used to reject or accept the nullhypothesis at some alpha level (probability of TypeI error) threshold, commonly set at 0.05 [14]. The re-jection of the null hypothesis at the p < 0:05 thresh-old tells the researcher that the means of the twoconditions are statistically different (i.e., signifi-cantly different). Table 2 shows the resulting p-valuecomparing the chance-corrected probability of targetdiscrimination of super-resolved versus original ima-gery at each target range using results obtained fromthe group of 10 subjects. Similarly, the paired t-testwas applied to the response times for super-resolvedversus original imager to generate p-values as tabu-lated in Table 3.For the probability of target discrimination of

super-resolved imagery versus original imagery, allp-values are less than 0.01, easily satisfying thesignificance threshold of p < 0:05. Therefore, thesubjects achieved significantly better probability oftarget discrimination with the super-resolved ima-gery at all target ranges. For response times, subjectsalso achieved significantly faster response times atall target ranges, except at 3:5m (which is close tosignificance) and at 6m (when subjects simply chosea random orientation for the original imagery).

E. Implications for Robotics Systems

The improvements in the probability of target discri-mination and response times using the TOD metho-dology shows the significant benefits provided bysuper-resolution for flash ladar imagery. The results

of this study, although not directlymeasuring the per-formance of robotics systems, is expected to be highlyindicative of the expected improvement of roboticssystems with the incorporation of super-resolvedflash ladar imagery, especially for semiautonomousrobotic applications where human perception isconcerned. It is likely necessary for future work todirectly assess the improvement both for semiauto-nomous and autonomous systems with the incorpora-tion of super-resolved flash ladar imagery. For futureassessment of autonomousapplications, objectiveme-trics, such as time to reach destination and number ofobstacles avoided, may be well suited. By repeatingacross multiple trials and assessing across multipleobstacle courses, an accurate performance compari-son could be made between robots with super-resolved flash ladar systems, regular flash ladarsystems, and those with traditional scanning ladarsystems. The authors expect that the results in thisstudy will be correlated with and indicative of the ex-pected improvement for robotic applications utilizingsuper-resolved flash ladar imagery.

4. Conclusion

Super-resolution image reconstruction comple-mented by wavelet preprocessing yields significantbenefits for flash ladar imagery. In the triangle orien-tation discrimination experiment, subjects achievedsignificantly higher accuracy at all investigated tar-get ranges with faster response times and reducedintersubject variability for super-resolved imagery.Complemented by super-resolution image recon-struction, the high frame rate, small size, and light-weight flash ladar sensors will be ideal for robotnavigation in urban indoor environments.

References1. T. C. Ng, SIMTech technical reports (2005), Vol. 6, No. (1),

13–18.2. Committee on Army Unmanned Ground Vehicle Technology,

“Technology development for Army unmanned ground vehi-cles,” Sandia Report (2002).

3. MESA Imaging, SwissRanger SR-3000 Manual (2006), http://www.mesa‑imaging.ch/.

4. G. Rosenbush, T. H. Hong, and R. D. Eastman, “Super-resolution enhancement of flash LADAR range data,” Proc.SPIE 6736, 673614 (2007).

Table 2. Computed p-Values for Each Target Range using Paired t -test Comparing the Probability of Target Discrimination of Super-Resolvedversus Original Flash Ladar Imagerya

Range (m) 3 3.5 4 4.5 5 5.5 6

p-value 0.00065 0.00592 0.00416 0.00663 0.000161 0.00003 0.00000aThose that achieve significance are in italics.

Table 3. Computed p-Values for Each Target Range using Paired t -Test Comparing the Response Times of Super-Resolved versus OriginalFlash Ladar Imagerya

Range (m) 3 3.5 4 4.5 5 5.5 6

p-value 0.00737 0.09008 0.04273 0.03824 0.01106 0.00217 0.67972aThose that achieve significance are in italics.


http://www.mesa-imaging.ch/




5. P. Vandewalle, S. Susstrunk, and M. Vetterli, “A frequency do-main approach to registration of aliased images with applica-tion to super-resolution,” EURASIP J. Appl. Signal Process.2006, 71459 (2006).

6. S. S. Young and R. G. Driggers, “Super-resolution image recon-struction from a sequence of aliased imagery,” Appl. Opt. 45,5073–5085 (2006).

7. N. Devitt, S. Moyer, and S. S. Young, “Effect of image enhance-ment on the search and detection task in the urban terrain,”Proc. SPIE 6207, 62070D (2006).

8. R. G. Driggers, K. Krapels, S. Murrill, S. Young, M. Thielke,and J. Schuler, “Super-resolution performance for under-sampled imagers,” Opt. Eng. 44, 014002 (2005).

9. P. Bijl and J. M. Valeton, “Triangle orientation discrimination:the alternative to MRTD and MRC,” Opt. Eng. 37, 1976–1983(1998).

10. S.S.Young,R.G.Driggers,andE.L.Jacobs,SignalProcessingandPerformanceAnalysis for ImagingSystems (ArtechHouse, 2008).

11. D. Anderson, H. Herman, and A. Kelly, “Experimental charac-terization of commercial flash ladar devices,” in Proceedingsof the International Conference of Sensing and Technol-ogy (Massey University, 2005), http://www‑ist.massey.ac.nz/conferences/icst05/proceedings/icst_2005.htm.

12. M. Price, R. D. Eastman, J. Kennedy, and T. Hong, “Trainingand optimization of operation parameters for flash LADARcameras,” in Proceedings of the IEEE International Conferenceon Robotics and Automation (IEEE, 2007), pp. 3408–3413.

13. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P.Flannery, Numerical Recipes in C: The Art of ScientificComputing (Cambridge U. Press, 1992).

14. E. L. Lehmann and J. P. Romano, Testing Statistical Hypoth-eses (Springer-Verlag, 2005).


http://www-ist.massey.ac.nz/conferences/icst05/proceedings/icst_2005.htm






Documents

Super-resolution for flash ladar imagery