Scene-Based Shack-Hartmann Wave-Front Sensing: Analysis and Simulation

Scene-based Shack–Hartmann wave-frontsensing: analysis and simulation

Lisa A. Poyneer

In many situations it is not possible for an adaptive optics system to use a point source to measure thephase derivative, such as imaging along slant paths through the atmosphere and observation of the earthfrom space with a lightweight optic. Instead, small subimages of the observed scene can be used in ascene-based wave-front sensing technique. This study presents three important advances in the un-derstanding of this technique. Rigorous analysis shows how slope estimation performance dependsprecisely on scene content and illumination. Scaling laws for changes in illumination are derived. Thetechnique, when applied to point sources, is more robust to detect size changes and background levelsthan current methods. © 2003 Optical Society of America

OCIS codes: 010.1080, 010.7350.

1. Problem Description and Background

Adaptive optics �AO� systems work by estimating andcorrecting for the phase aberration along the opticalpath. The performance improvements obtainedwith AO can be quite dramatic.1 Normally a deriv-ative of the phase is measured, and the phase isreconstructed from those measurements. A pointsource of light is in most cases used as the reference.

There are many situations, however, when a pointsource is not available but AO correction is desirable.Of prime interest are two situations. The first isthat imaging conducted along horizontal or slantpaths through the atmosphere could be improvedwith AO correction for the turbulent atmosphere.Note that because in this paper we are concernedwith how to measure the phase, not how to correct it,we will not discuss whether scintillation or refractionwill inhibit the performance of multiwavelength AOalong slant paths. The second situation is the ob-servation of the Earth from space with lightweightoptics. These optics are less expensive but can suf-fer from time-varying aberrations due to thermal ef-fects and vibration.2 AO could correct these phaseaberrations. In both cases, observations of an ex-tended scene do not provide a point source. Instead

The author �e-mail [email protected]� is with Lawrence Liver-more National Lab, Livermore, California.

Received 14 March 2003; revised manuscript received 1 July2003.

0003-6935�03�295807-09$15.00�0© 2003 Optical Society of America

of trying to create an artificial point source, the sceneitself can be used to make phase estimates.

There are two major ways to use images to esti-mate phase. The first is called phase retrieval.3This method uses multiple images, some with aknown additional phase aberration, to iteratively de-termine the phase. This method is commonly usedin telescope alignment. In addition, phase-diversitymethods can be used to correct for atmospheric aber-rations with image post processing.4 These methodsare normally quite computationally intensive. In-stead, we chose to investigate in detail a secondapproach that is very similar to current point-source-based AO systems. In fact, the only difference ishow the slope estimates are computed.

The second option is the use of small images asproduced by a Shack–Hartmann sensor array. In-stead of forming a spot �an image of the point-sourcereference�, the lenslets form small images of the ob-served scene. The subimage is at lower resolutionbecause of the smaller size of the subaperture. Eachsubimage will have a field of view that is restricted bytwo main considerations. First, the field will need tobe large enough to contain scenes capable of beingused for slope estimation given the subaperture dif-fraction limit. Second, the field must be smallenough that the subimages will not overlap on thewave-front sensing �WFS� camera. This maximumsize is set by the physical separation of the lenslets inthe WFS array and the magnification. The exactspecifications are a systems-design issue, and mostlikely will vary depending on the remote imagingscheme. Each of these subimages will be shifted by

10 October 2003 � Vol. 42, No. 29 � APPLIED OPTICS 5807

a small amount due to the local phase distortionacross each lenslet, just as a point source is. Theproblem is then how to best estimate the shifts be-tween the images.

This Shack–Hartmann approach, which we refer toas scene-based wave-front sensing �SBWFS�, is usedsuccessfully at the National Solar Observatory5 andother solar observatories. For the case of solar gran-ulation, the scene has very specific characteristics.For the special case of low-contrast solar granulation,the error standard deviation �x of the slope was es-timated to be proportional to the background noisestandard deviation divided by the image contrast.5However, no rigorous performance analysis of wavefront slope estimation based on arbitrary scene con-tent and illumination has been published. A sens-ing system could look at an arbitrary scene, whetherit contains buildings in a city or a road in the desert.The illumination, which is due to time of day, angle ofobservation, and atmospheric characteristics, ishighly variable. Intuitively, the performance of agiven scene depends on the content of the scene aswell as the illumination characteristics. The morefeatures and high-spatial-frequency content and themore light, the better the performance.

This intuitive understanding needs to be rigorouslyquantified for several reasons. First, a system canbe specifically designed to meet performance criteriagiven expected illumination and scene content. Sec-ond, as the system is built and tested, analysis en-ables confirmation of performance. Finally, real-time knowledge of scene quality could be used tooptimize AO system performance by adjusting sys-tem parameters, such as frame rate or changing thescene that is used. For all of these reasons, detailedanalysis of the scene-based slope estimation problemhas been carried out and is presented here. Simu-lation confirms the analytic predictions, showing thatSBWFS is a viable technique that will enable AO tobe used in a broad range of new situations.

2. Slope Estimation from a Scene

As described above, multiple subimages of the ob-served scene are formed for use on the WFS camera,just as multiple images of the point source are. Thebasic concept of SBWFS is to then compare the imagein a given subaperture to a reference image and es-timate the shift between the two images. This shiftcould be determined by cross correlating a referenceimage with each subaperture image.6 However,many different methods exist for aligning images.7Several were analyzed as candidates for obtainingslope information. This section describes the modelof the subimages and discusses possible algorithmsfor slope estimation. Based on both performancewith noise and computational simplicity, periodic cor-relation was chosen as the preferred method.

A. Definition of Terms

Consider two discrete images �made of pixels on theWFS camera� that are formed by distinct subaper-tures. One image is the reference, r�m, n�, which

will be used in comparison with all the other subap-ertures. The second image is s�m, n�. The two sig-nals are shifted slightly from each other by x0 and y0samples, which are not necessarily integer amounts.For now, consider the signals as being infinite inextent, and as sampled versions of the same under-lying continuous signal. The sample interval is D.Given the high-resolution base signal i�x, y�, the ref-erence signal is

r�m, n� � i�mD, nD�. (1)

The subaperture image is assumed to be simply ashifted version

s�m, n� � i�mD � x0 D, nD � y0 D�. (2)

The shifts we desire to find are x0, y0. Because x0and y0 are in general not integers, the expression

s�m, n� � r�m � x0, n � y0� (3)

has no meaning except in the model described above.We also have the frequency domain counterparts ofthe above equations. Where I� fx, fy� is thecontinuous-time Fourier transform of i�x, y�, we canwrite expressions for the discrete-time Fourier trans-form of the other signals �which use frequency vari-ables �x, �y�

R��x, �y� �1

T 2 I��x

D,

�y

D� , (4)

S��x, �y� � R��x, �y�exp��j2�� x0�x � y0�y��. (5)

In reality, the two images r and s are of small finiteextent N N pixels and I is not necessarily bandlimited. Converting the above equation for use withdiscrete Fourier transform produces

S�k, l� � R�k, l�exp��j2�� x0 k � y0 l �

N � . (6)

B. Derivation of Algorithm

Based on this model, three possible methods to de-termine x0 and y0 were analyzed. They aremaximum-likelihood estimation, deconvolution, andcorrelation. These methods are applied to the finite,discrete signals obtained from the WFS camera.The reference subaperture image r�m, n� and thesubaperture image for slope estimation s�m, n� areboth N N pixels. Note that N may be preferen-tially a power-of-2 for some of the algorithms.

Because the two images are in fact noisy realiza-tions, we can regard the search for x0 and y0 as anonrandom parameter estimation problem. Given anoise model for both images and the actual scenecontent, the probability distribution for each could inprinciple be derived. The maximum-likelihood esti-mate is then determined. This technique has beenapplied to point-source WFS.8 However, this esti-mator is directly dependent on scene content, mean-ing it would have to be recalculated for every scene.Furthermore, since this is not a linear function of the

5808 APPLIED OPTICS � Vol. 42, No. 29 � 10 October 2003

nonrandom parameters x0 and y0, producing the es-timator is nontrivial computationally. This methodof estimation was therefore discarded as being toocomputationally intensive.

The second method is deconvolution. Thismethod exploits the frequency-domain relationshipbetween the two images �Eq. �6��. With use of thediscrete Fourier transform, S�k, l� is known, and ifR�k, l� is nonzero at all k, l, division produces

F�k, l� � exp��j2�� x0 k � y0 l �

N � . (7)

If x0, y0 are integers, the signal f �m, n� �obtained byinverse transforming the above� is simply a unit im-pulse at x0, y0. If x0, y0 are not integers, f �m, n� is ashifted sampling of a unit impulse. Based on thissignal, the shifts can be estimated.9 This deconvo-lution method is sensitive to noise, particularly whenthe spatial frequency content of the images is mainlylow frequency. Tests of this method in our applica-tion showed it to be highly susceptible to noise.

The final method under consideration is correla-tion. This method is an implementation of a mini-mal mean-squared error �MSE� metric. The shiftwith the least-squared difference between the twoimages r and s is the best answer. Formulated ex-plicitly, the MSE e�m, n� of the two images at dis-placement m, n is

e�m, n� � � �r�i � m, j � n� � s�i, j��2�� N � m��N � n��1, (8)

where the summation is, for m � 0, from i � m to i �N � 1, and for m � 0, from i � 0 to i � N � 1 � m.This holds likewise for n with index j. Note thate�m, n� is defined only over the range ��N � 1� � m,n � N � 1. Expanding the terms produces

e�m, n� � � r�i � m, j � n�2 � s�i, j�2

� 2r�i � m, j � n�s�i, j�� N � m��N � n��1. (9)

The last term in the summation in Eq. �9� looks likea correlation. But simply calculating the correlationis not a shortcut to calculating this metric. Becauseof the limits of summation, all the terms are depen-dent on m and n. Minimizing the MSE is not equiv-alent to maximizing the correlation between r and s,even when it is calculated with energy normalization.

Instead, the finite signals can be treated as a singleperiod of an infinite periodic signal �just as for the

discrete Fourier transform.� Now the limits of sum-mation are constant for all values of m and n

e�m, n� � �i�0

N�1

j�0

N�1

r�i � m, j � n�2 � s�i, j�2

� 2r�i � m, j � n�s�i, j��N�2. (10)

Owing to the periodicity of the signals, as one end ofthe signal moves away, it wraps around from theother side. In this case the formula simplifies dra-matically as the two energy terms remain constant.The MSE equation now becomes

e�m, n� � � i�0

N�1

j�0

N�1

r�i � m, j � n�s�i, j�, (11)

which is exactly the correlation of the two signalscalculated with periodic convolution. Minimizingthe MSE �as given by Eq. �11�� is the same as maxi-mizing the correlation. This correlation can be cal-culated quickly in the frequency domain by use ofconjugation and the correlation theorem.

Computing the exact MSE for all possible overlapsis O�N4� in FLOPs, where N is the number of pixelsalong each dimension of the square image. The pe-riodic correlation method can employ fast fouriertransforms �FFT�, so it is O�N2 log2N�. Except forthe case where the shifts are known to be small andonly a few points of the MSE must be calculated, theperiodic correlation method is significantly faster.For a 16 by 16 pixel subaperture image, the periodiccorrelation requires a factor of 16 times less compu-tation than calculating the MSE for all possible over-laps. However, the periodic correlation technique issusceptible to errors due to the wraparound of pixels.For low-quality images there is a trade-off: MSE pro-vides better estimates but with more computation.

Despite this wraparound effect, the periodic corre-lation technique can work just as well as the exactMSE method. As shown in Fig. 1, Monte Carlo sim-ulation demonstrates the comparable estimate errorand standard deviation for the same image with boththe MSE technique and periodic correlation. Forhigh-quality images, this overlap effect is negligible,and both methods perform equally well. For poorerquality images, the non-common image content at theedges has a significant effect.

It is important to note that the answer we want outof this technique is not the whole-pixel shift with thebest value of the metric, but rather an estimation ofthe best sub-pixel shift. In the general case the shiftwill not be a whole-pixel amount. The correlation isactually just the scaled sampling of the autocorrela-tion of the base signal sampled at the shift x0 and y0

C�m, n� �1

D2 C� x, y��x�mD�x0 D,y�nD�y0 D. (12)

To obtain the true maximum of the function, we sim-ply need to interpolate to estimate the continuoussignal C�x, y� and take its maximum value. Para-


bolic interpolation is used, though another curve,such as a Gaussian, could be fit to the peak. Assumethat the maximum of the discrete signal C�m, n� islocated at integers � x, y�. Then the followingequation fits a parabola to the closest points aroundthe peak and gives the shift estimate:

x0 � x

�0.5�C� x � 1, y� � C� x � 1, y��

C� x � 1, y� � C� x � 1, y� � 2C� x, y�.

(13)

The estimate of y0 is obtained in an analogous fash-ion.

In summary, the best method for estimating theshift between two subimages uses periodic correla-tion. This is done by calculating the correlation withFFTs and doing parabolic interpolation around thepeak location to determine the best sub-pixel shiftestimate. We have elected to use a reference subap-erture to provide the new reference subimage at ev-ery time step. This ensures that any dynamicchanges in the scene will be compensated for. Thereare two complications to this. First, at any time stepthere is no knowledge of the true tip and tilt, becauseall slope estimates are measured relative to the offsetof a single subaperture. Therefore the drift of thissubaperture through time should be followed to ob-tain tip and tilt information. Second, using a new,noisy reference at each time step increases theamount of noise in the system. Compared with afixed �nonrandom� reference image, the noise vari-ance when using a new reference every step is two tothree times greater. However, if there is any dy-namic change in scene content, this fixed referencemay provide poor quality estimates because of differ-ences in the images. In the general case we expectchanges in the scene. The performance of thismethod of shift estimation is rigorously analyzed be-low. Simulation is then used to confirm the analy-sis.

3. Performance Analysis and Simulation Results

A. Motivation

In AO systems, noise on the wave-front sensor man-ifests itself as noise on the slope estimation. Thiserror then propagates through to the compensatedphase. Scene-based slope estimation can be ana-lyzed by use of this model, where image quality aswell as photon noise leads to noise on the slope esti-mates. Image content will vary to a large degree,with some scenes being full of features, such as build-ings or vehicles, and others being relatively uniformsuch as a barren patch of land. Differences in imagecontent will produce images that are better or worsefor estimation. The total amount of light and back-ground scatter will also vary with angle of viewing,time of day, air quality, etc. These characteristicswill affect the noise level and change the performanceof a specific scene.

The bias and variance of the slope estimate are ofinterest. The ideal scene would produce an unbi-ased estimate with very low �or no� variance. This isof course not true in the general case, and both biasand variance depend on the scene, the noise level, andthe amount of shift between the subimage and itsreference. This Section analyzes the characteristicsof the slope estimation based on image content, illu-mination and actual image shift. This analysis pro-vides both a measurement of performance for anarbitrary case, as well as scaling laws that describehow performance changes with specific parameters.

Monte Carlo simulations were used as a techniquefor verifying the analytic results. In these simula-tions a large diffraction-limited image �obtained fromcommercial satellite imagery� was used. The sub-pixel shifted image was obtained by convolution.The larger signal was windowed down to simulate thefield stop and the real drift of the image within thefield. Given a specific illumination profile, imagecontent, and noise model, random realizations of theimages were generated. Tens of thousands of trials

Fig. 1. Periodic correlation has competitive performance with the MSE method for a good image. In this case, Monte Carlo simulationresults for slope estimation of a shift in the x direction are shown: �a� Error of estimation in both cases is small, �b� the standard deviationof the estimates are comparable.


on the same set of conditions were conducted, and theresults were statistically analyzed.

B. General Case

As presented above, slope estimation involves com-puting the cross correlation of two subaperture im-ages �r�m, n� and s�m, n��, finding the maximum, andthen using that maximum value and the two neigh-boring values each to determine the estimate of theshifts x0 and y0 via parabolic interpolation. The ran-dom vector C�m, n� represents the cross-correlationfunction of these two images. Specifically, this pe-riodic correlation

C�m, n� � i

j

r�i � m, j � n�s�i, j� (14)

can be computed with FFTs. The maximum of thecorrelation will be �for whole-pixel shifts� at exactlyC�x0, y0�. For sub-pixel shifts, the maximum of thiscorrelation function is at � x, y�. We will assumethat the maximum of C�m, n� is within half a pixel ofthe actual shift. For some degenerate images, it willnot be. For a single estimate of the x slope, parabolicinterpolation is used. This requires the discretemaximum C� x, y� �as opposed to the true maximumof the continuous correlation� and the two pointsbracketing it, C� x � 1, y� and C� x � 1, y�. Fornotational simplicity, these three points will be re-ferred to as C0 for the maximum and C�1 and C1 forthe neighbors. The estimate of the shift is then re-written into Eq. 13 as

x0 � x �0.5�C�1 � C1�

C�1 � C1 � 2C0. (15)

This expression is very difficult to analyze. Becauseit involves division of random variables, full knowl-edge of the probability distributions of C�1, C0, andC1 is required to characterize the resulting randomvariable. Preferably, knowledge of just the means�m�1, m0, m1�, variances ��1

2, �02, �1

2� and covari-ances ��1,0

2, ��1,12, �0,1

2� of the random variablesC�1, C0, C1 will be adequate to evaluate the perfor-mance of the estimator. This is true if the estimatorx0 is approximated as a linear combination of C�1, C0,C1 instead of being a quotient. The tangent plane tothe function is constructed at the point �C�1 � m�1,C0 � m0, C1 � m1�. The function f �C�1, C0, C1� isapproximated as f �C�1, C0, C1�. This is doneby taking partial derivatives and evaluating them atthe correct location. Where the partial derivative off �C�1, C0, C1� with respect to variable Ci is fi�C�1, C0,C1�, the approximation is

f �C�1, C0, C1� � f �m�1, m0, m1�

� �C�1 � m�1� f�1�m�1, m0, m1�

� �C0 � m0� f0�m�1, m0, m1�

� �C1 � m1� f1�m�1, m0, m1�. (16)

Evaluating the above expression produces the linearapproximation for the slope estimate:

x0 � x � �C�1�m1 � m0� � C0�m�1 � m1� � C1�m0

� m�1� � 0.5�m�1 � m1��m�1 � m1 � 2m0��

� �m�1 � m1 � 2m0��2. (17)

Because this is a linear combination of random vari-ables, the mean and variance can be determined withknowledge of only the means and variances of itscomponents. Most terms cancel to make the expec-tation simply

E� x0� � x �0.5�m1 � m�1�

m�1 � m1 � 2m0. (18)

The variance can also be explicitly calculated andafter simplification is

�x2 � ��1

2�m1 � m0�2 � �0

2�m�1 � m1�2

� �12�m0 � m�1�

2

� 2�m1 � m0��m�1 � m1��1,02

� 2�m1 � m0��m0 � m�1��1,12

� 2�m�1 � m1��m0 � m�1��0,12�

� �m�1 � m1 � 2m0��4. (19)

The means, variances, and covariances �e.g., m0, �12�

that appear in the above equations can be easily cal-culated from the statistical models of the images.The explicit formulas for these terms are given in theAppendix. This long expression is not particularlyinformative on its own. As the following discussionshows, it can be drastically simplified for specific use-ful cases. The results provide powerful descriptionsof estimation performance.

C. Zero-Shift Case

A special case worth considering is when the actualshift between the two images is zero. In a closed-loop system, the image shift will be driven towardnull. This simplification also enables easier analy-sis of slope estimation behavior as illumination con-ditions change. In this zero-shift case, the twosubimages have identical distributions. Thereforem�1 � m1 and ��1

2 � �12. This reduces the approx-

imation of the estimate �Eq. �17�� to be

x0 � x �C�1 � C1

4�m1 � m0�. (20)

In this special case, the correlation is actually anautocorrelation, so the peak will be at 0 and themeans and variances of C�1 and C1 will be equal.Therefore the estimate is unbiased

E� x0� � 0. (21)


Most of the terms in Eq. �19� cancel, reducing theexpression for the variance to

�x2 �

�12 � ��1,1

2

8�m0 � m1�2 . (22)

The most important term in this equation is the de-nominator term �m0 � m1�. As described above, m0is the expected value of the maximum of the correla-tion function �C0� and m1 is the expected value onepixel to the side �C1�. The �m0 � m1� term is then ameasure of the sharpness of the correlation peak.The sharper this peak, the lower the error variance.The correlation function is paired with its Fouriertransform partner: the power spectral density.The more impulse-like the correlation function�hence the sharper the peak� the broader the fre-quency content of the image. This is consistent withthe notion that images with more high-frequency con-tent perform better.

In the zero-shift case, all that is needed to calculatethe error variance of the estimate is knowledge of thesubimage statistics. By use of the parameters thatare defined in the Appendix �see Eq. �A5–A7��, theperformance of any image can be quickly calculated.When done for a wide range of images, the predictedslope estimate variance �x

2 reveals substantial vari-ation in image quality. Some scenes have muchlower variance for estimation along one axis than theother by a significant amount. For example, a scenewith a road that runs horizontally across the subim-age will be a much better estimator in the y axis thanin the x axis. This is because the image is self-similar for shifts along the x axis, as one part of theroad looks much like another.

The above analysis showed the estimator to be un-biased and the error variance predictable in the zero-shift case. Monte Carlo simulations confirm thesepredictions. For a wide range of images, the slopeestimate mean is extremely small for a wide range ofimages, confirming the estimator to be unbiased.Fig. 2 shows the predicted error standard deviation�x �by Eq. �22�� compared with the results from sim-ulation. For low values of �x, the prediction ishighly accurate. It starts to under estimate forlarger values because of the linear approximationthat was used to produce a tractable expression forestimator variance.

D. Illumination Changes

A metric that differentiates between arbitrary im-ages was derived above. It is also of interest how thequality of a specific image varies with changing typesand levels of illumination. Exposure time and totalamount of light are the first concern. System framerate is important, as there is a significant designtrade-off between the benefits of high speeds �e.g.,faster correction results in reduced temporal errors�and the detriments �e.g., reception of fewer photonsleads to more noise.�

To capture the effects of changing light levels, thepixel values of each subimage are modelled as inde-

pendent Poisson random variables with parametersf�r�m, n�, f�s�m, n�. The parameters � are normal-ized from 0 to 1 and multiplied by a scale factor f,which represents the number of photons received bythe brightest possible pixel. This is a mathematicalconvenience, but also could reflect real estimations ofthe physical object’s albedo. The means and vari-ances of C�1, C0, C1 are calculated as given in theAppendix. Insertion of these values shows that theslope estimate expectation is independent of lightlevel and depends only on the image content. This istrue regardless of the amount of shift. The new re-lation for the error variance in the zero-shift case is

�x2� f � �

�12 � � f � 1� f �1m1 � ��1,1

2

8f �m0 � m1�2 . (23)

This is very close to the basic expression for zero-shifterror variance �Eqn. �22��. For a large number ofphotons f �i.e., bright light� the � f � 1� f�1 term in thenumerator is approximated as 1, producing a scalinglaw for the standard deviation of the estimate:

�x� f � �1

�f

��12 � m1 � ��1,1

2�1�2

2�2�m0 � m1�. (24)

In the bright-light case, the signal-to-noise ratio�SNR� is simply proportional to �f, where f is themaximum amount of light per pixel. The standarddeviation of the estimate follows the same inversepower law to the SNR as quad-cell centroiding with apoint source does.10 Though the constant of the re-lationship may be different �and is image dependent�,this method of wave-front sensing is statisticallyequivalent to the traditional approach with a pointsource and centroiding. By use of the same � f �1� f�1 � 1 approximation, in the general case forshifts of an arbitrary amount, this inverse relation-ship to the SNR still holds. This means that achange in total amount of light simply results in the

Fig. 2. For the zero-shift case, the bias and standard deviation �stdev� of the estimate can be explicitly calculated from the imageitself a priori. The predicted standard deviation � vs. simulationresults for a shift of zero. For high-quality images, the predictionis extremely accurate. It starts to under predict for poorer im-ages.


estimate standard deviations for all subapertures tobe scaled by a known constant.

A second very useful way to look at the problem isto break the subimage down into two components:the actual image and the uniform background. Thismodel is appropriate for paths through the atmo-sphere, when scattering increases the backgroundlevel. This can be due to long paths or high partic-ulate levels.11 In this case the image pixels are pa-rameterized by b � f��m, n�, where b is the level ofbackground light. Again, for any shift, the estimateexpectation is independent of background level andillumination amount. After some algebraic simpli-fication, the estimate error standard deviation in thezero-shift case is

where N is the total number of pixels in a singlesubimage and t is another image statistic �see Appen-dix A.� The sharpness of the correlation peak is stillthe dominant term in this expression.

The advantage of these scaling laws is that, basedon a single copy of the subimage at a known lightlevel, the behavior of that image over a broad range ofconditions can be predicted. Figure 3 shows simu-lation results for a specific image over a range of totallight and amount of background. The top panelshows the effect of reducing the exposure time; thebottom shows the effect of increasing amounts ofbackground. In both cases, the predictions �usingEqs. �23� and �25�� based on a single copy of the imageare a very good fit to the actual performance. The

scaling approximation �Eq. �24�� has significant errorat very low light levels, which is to be expected due tothe � f � 1� f�1 � 1 approximation.

These laws aid in system design. In terms ofchoosing the frame rate, performance falls off withthe inverse of the SNR. Background light can bevery detrimental. When the amount of backgroundlight begins to exceed the image signal, the standarddeviation increases rapidly, though the estimatemean remains the same. In the case shown in Fig.3, similar performance is obtained for a no-background image with a maximum of 100 photonsper pixel as for a background level of 250 photons perpixel and an image level of another 250 photons.Five times as much total light is required in this case

when the background scatter is equivalent to the im-age content.

4. Computational Cost Estimates

Better AO correction drives system design to moreactuators and faster rates, which are limited by thetotal amount of light available. But more pixels andmore subapertures lead to an increased computa-tional burden. A balance must be struck betweencomputational costs and desired correction levels.As an FFT-based algorithm, SBWFS computationscales as O�np

2log np�, where the number of pixels npacross the subaperture is a power of 2. This compu-tation must be done for each of the ns subapertures,making the total cost O�nsnp

2 log np�. These slope

Fig. 3. Scene performance varies with the total amount and type of illumination. Simulation results are shown for the x- and y-slopeestimate in the zero-shift case. In both panels the open symbols are simulation results. The solid curves are exact calculations of scenequality. The dashed curves are predictions based on the scaling law �Eq. �24�� and a single copy of the image: �a� Variation in totalamount of light. Reduction in the total amount of light causes a fall-off inversely proportional to the SNR. �b� Increasing backgroundand constant total amount of light. Performance falls off rapidly once the amount of background light is equal to or greater than imagelight. For this image, 100 photons per pixel of image and no background gets comparable performance to 250 photons background and250 photons image. St dev is standard deviation.

�x�b, f � ��Nb2 � 2bf 2�m0 � m2 � tf �1� � f 3��1

2 � � f � 1� f �1m1 � ��1,12��1�2

2�2f 2�m0 � m1�, (25)


estimates are then reconstructed to obtain the phase.For small numbers of actuators na, the vector-matrix-multiply method is fine. But it scales as O�na

2�.For larger systems a faster algorithm, such as Fou-rier transform reconstruction �FTR�,12 is preferable.FTR scales as O��na�2�log na�.

The dominant computational term is the slope es-timation, which becomes enormous for large ns. Ex-act FLOPs calculations are given in Table 1 for arange of reasonable schemes using FTR. For highcontrol rates on large systems, parallel processing ofthe slopes will be necessary. However, a low-orderAO system, with 14 subapertures across the pupildiameter �156 total�, running at 10 Hz with 16 16pixel subapertures would require a 16 MFLOPs�mega-floating point operations� per second for sens-ing and reconstruction. This is a very reasonablecomputational burden.

5. Scene-Based Wave-Front Sensing for Point Sources

The preceding sections have shown that SBWFS is arobust method. It can be applied to arbitraryscenes, with both estimate mean and variance pre-dictable a priori. Operating on null, it exhibits nobias and the same inverse-power-law performancethe same as using a point source. For an arbitraryshift, the slope estimate mean is insensitive to back-ground light. Why not use it for wave-front sensingwith a point source? Current methods calculate thecenter of mass of the spot formed by the pointsource.13 SBWFS has a couple of particular advan-tages to these approaches.

In the general case, many AO systems can havebackground light on the WFS. This can be due tothe internal WFS camera characteristics or simply toscattered light. It may be very difficult to charac-terize and correctly remove the background. Evenlow levels of background can cause a centroiding al-gorithm to produce very poor answers. As shownabove, any uniform background level is irrelevant tothe slope estimate expectation in SBWFS. There-fore no detailed background subtraction is necessary,and a time-varying level of background light will notaffect system performance.

SBWFS could address problems specific to astro-nomical AO systems with the quad-cell �2 2 pixelsubaperture� arrangement. The SBWFS wouldhave to use 4 4 pixels to have enough points to

make a correlation peak to interpolate. There aretwo well-documented situations where the quad-cell method suffers from extra error, both having todo with the size of the spot. As the size of the spotchanges �which it does dynamically owing to chang-ing atmospheric conditions� the gain of the WFSmeasurement also changes, when it is calculatedwith the quad-cell centroid algorithm. This mat-ter, and how to compensate for it, has been stud-ied.14 Spots, no matter their size, are good scenes.Analysis shows that within the linear range of theWFS, the SBWFS algorithm is insensitive to thespot size changes. Note that if the spot is toosmall, the WFS is not linear, and neither methodwill work well.

Second, and somewhat more subtly, AO systemscan be run off null with the use of reference centroids.Once calibrated, a specific subaperture is driven to-ward a non-zero shift. If the spot size changes, thecentroiding algorithm will no longer accurately mea-sure this offset, causing problems with AO operation.This has occurred when Uranus was used instead ofa guide star in the Keck AO system.15 Because SB-WFS is insensitive to this spot size change it wouldenable this kind of operation.

It must be noted that a disadvantage of using SB-WFS is the extra computational cost. In the astro-nomical case SBWFS requires 4 times more FLOPsthan a 4 4 centroider and 30 times more FLOPsthan the quad-cell algorithm. If the AO system haslimited computational resources, the extra computa-tion for SBWFS may cause a re-duction in system rate and hence greater temporalerrors.

SBWFS has the potential to be an alternative slopeestimation algorithm for point sources. Provided asuitable reference is available, SBWFS could produceaccurate, consistent slopes as conditions, such as spotsize and background level change, unlike current cen-troiding methods. The author is currently under-taking a detailed comparison of centroiding andSBWFS for point-source sensing in astronomical andvision AO systems.

6. Conclusions

This paper has presented several important advancesin the understanding of SBWFS. Scene perfor-mance as a slope estimator is shown to be entirelydependent on the scene content and illuminationtype. In the general case, slope estimate bias is fullypredictable a priori and is independent of scene illu-mination and camera characteristics. With a Pois-son statistics model, the slope estimation is shown tofollow the same inverse power law in SNR for errorstandard deviation as point sources do. The SBWFStechnique can be applied to point sources, potentiallyproviding a WFS method that is more robust to spotshape, size, and background levels. The materialpresented here enables the design and use of AOsystems that can use an arbitrary scene to do Shack–Hartmann-based wave-front sensing instead of apoint source.

Table 1. Total kFLOPsa per Cycle for Slope Estimation and PhaseReconstruction

Step 156 Subaps.b 716 Subaps.b 3024 Subaps.b

Slope 8 8 300 1,370 5,810Slope 16 16 1,600 7,330 31,000Slope 32 32 7,990 36,700 155,000Phase reconc 61.4 102 492

akFLOPs, kilo-floating point operations.bSubaps., subapertures.cRecon, reconstruction.


Appendix A: Calculation of the Statistics of theCorrelation Function

To determine the mean and variance of the estimator,all that is needed is knowledge of the means andvariances of the individual pixels of the WFS camerathemselves. In the most general case, both imagesare modelled as independent vectors of independentrandom variables. For the two vectors r�i, j�, s�i, j�,each element has mean mr�i, j� or ms�i, j� and vari-ance �r

2�i, j� or �s2�i, j�. Let Ck be the product of the

two vectors at offset k in the x direction, namely:

Ck � i

j

r�i � k, j�s�i, j�. (A1)

The mean value is

E�Ck� � mk � i

j

mr�i � k, j�ms�i, j�

(A2)

and the variance is

Var�Ck� � �k2 �

i

j��r

2�i � k, j��s2�i, j�

� �r2�i � k, j�ms

2�i, j�

� mr2�i � k, j��s

2�i, j��. (A3)

The covariance of any two values of the correlationfunction is

Covar�Ck, Cl� � �k,l2 �

i

j��s

2�i, j�mr�i � k, j�

� mr�i � l, j� � �r2�i, j�ms�i

� k, j�ms�i � l, j��. (A4)

If the reference is fixed �nonrandom�, the above anal-ysis still applies. The variance �r

2�i, j� now simplyequals zero in the above formulas.

Given that the images are formed from light re-ceived by an optical system, the appropriate choicefor the random variables is that they have Poissondistributions. This means that the expectation andvariance of each random pixel are equal, namely mr�i,j� � �r

2�i, j� � �r�i, j�. Plugging these terms into theabove equations generates simple summations of �parameters to obtain the statistics. The mean valueof Ck is

mk � i

j

�r�i � k, j��s�i, j�, (A5)

the variance is

�k2 �

i

j�r�i � k, j��s�i, j�

� �1 � �s�i, j� � �r�i � k, j��. (A6)

The covariance of Ck and Cl is

�k,l2 �

i

j��s�i, j��r�i � k, j��r�i � l, j�

� �r�i, j��s�i � k, j��s�i � l, j��. (A7)

One other term that will be used is t, which is the sumof the parameters ��i, j�. This model can be easilyexpanded to account for gain factors, read noise, andbackground light.

The work detailed herein is part of a larger pro-gram, and the author has benefitted from interac-tions with her colleagues J. Brase, L. Flath, D. Gavel,E. Johansson, K. LaFortune, R. Sawvel, and C.Thompson. This work was performed under theauspices of the U.S. Department of Energy by theUniversity of California, Lawrence Livermore Na-tional Laboratory under contract No. W-7405-Eng-48. The document number is UCRL-JC-151642.

References1. P. Wizinowich, D. S. Acton, C. Shelton, P. Stomski, J. Gath-

right, K. Ho, W. Lupton, K. Tsubota, O. Lai, C. Max, J. Brase,J. An, K. Avicola, S. Olivier, D. Gavel, B. Macintosh, A. Ghez,and J. Larkin, “First light adaptive optics images from theKeck II telescope: a new era of high angular resolution im-agery,” Publ. Astron. Soc. Pac. 112, 315–319 �2000�.

2. P. C. Chen, C. W. Bowers, D. A. Content, M. Marzouk, andR. C. Romeo, “Advances in very lightweight composite mirrortechnology,” Opt. Eng. 39, 2320–2329 �2000�.

3. M. G. Lofdahl and G. B. Scharmer, “Wavefront sensing andimage restoration from focused and defocused solar images,”Astron. Astrophys. Suppl. Ser. 107, 243–264 �1994�.

4. R. G. Paxman, J. H. Seldin, M. G. Lofdahl, G. B. Scharmer, andC. U. Keller, “Evaluation of phase-diversity techniques forsolar-image restoration,” Astrophys. J. 466, 1087–1099 �1996�.

5. T. R. Rimmele, “Solar adaptive optics,” in Adaptive OpticalSystems Technology, P. L. Wizinowich, ed., Proc. SPIE 4007,218–231 �2000�.

6. T. R. Rimmele, O. von der Luehe, P. H. Wiborg, A. L. Widener,R. B. Dunn, and G. Spence, “Solar feature correlation tracker,”in Active and Adaptive Optical Systems, M. A. Ealey, ed., Proc.SPIE 1542, 186–193 �1991�.

7. L. G. Brown, “A survey of image registration techniques,” ACMComp. Surveys 24, 325–376 �1992�.

8. M. A. van Dam and R. G. Lane, “Wave-front slope estimation,”J. Opt. Soc. Am. A 17, 1319–1324 �2000�.

9. H. Foroosh, J. B. Zerubia, and M. Berthod, “Extension of phasecorrelation to subpixel registration,” IEEE Trans. Image Pro-cess. 11, 188–200 �2002�.

10. G. A. Tyler and D. L. Fried, “Image-position error associatedwith a quadrant detector,” J. Opt. Soc. Am. A 72, 804–808�1982�.

11. W. L. Wolfe and G. J. Zissis, eds., The Infrared Handbook,revised ed. �Office of Naval Research, Washington, D.C.,1978�.

12. L. A. Poyneer, D. T. Gavel, and J. M. Brase, “Fast wave-frontreconstruction in large adaptive optics systems with use ofthe Fourier transform,” J. Opt. Soc. Am. A 19, 2100–2111�2002�.

13. J. W. Hardy, Adaptive Optics for Astronomical Telescopes �Ox-ford University, New York, 1998�.

14. J.-P. Veran and G. Herriot, “Centroid gain compensation inShack–Hartmann adaptive optics systems with natural or la-ser guide star,” J. Opt. Soc. Am. A 17, 1430–1439 �2000�.

15. I. de Pater, S. G. Gibbard, B. A. Macintosh, H. G. Roe, D. T.Gavel, and C. E. Max, “Keck adaptive optics images of Uranusand its rings,” Icarus 160, 359–374 �2002�.


Documents

Scene-Based Shack-Hartmann Wave-Front Sensing: Analysis and Simulation